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Executive  Summary 


Current  airspace  structure  is  rigid  and  does  not  allow  for  dynamic  resectorization  of  airspace 
boundaries.  Dynamic  resectorization  is  adaptive  and  can  efficiently  handle  heavy  traffic 
situations,  shifting  weather  conditions,  status  changes  in  special  use  airspace,  and  user-preferred 
routes.  Dynamic  resectorization  has  the  potential  to  reduce  aircraft  delays,  decrease  fuel 
consumption,  and  lower  operating  costs  for  the  airline  industry.  The  potential  benefits  for  Air 
Traffic  Control  Specialists  (ATCSs)  are  to  offset  heavy  workload  and  reduce  coordination  and 
communications.  However,  dynamic  resectorization  may  be  disruptive  and  could  have  negative 
consequences  for  controller  situational  awareness  and  performance.  This  report  describes  a  real¬ 
time  human-in-the-loop  simulation  study  designed  to  investigate  a  specific  approach  to 
implementing  dynamic  resectorization  between  two  adjacent  Air  Route  Traffic  Control  Centers 
(ARTCCs). 

The  objective  of  the  study  was  to  examine  the  impact  of  inter-facility  dynamic  resectorization  on 
ATCSs’  workload,  communication,  situational  awareness,  control  strategies,  and  performance. 
As  a  preliminary  investigation,  the  scope  of  the  study  was  limited  to  lateral  boundary 
adjustments  and  specific  traffic  situations  that  should  benefit  the  most  from  dynamic 
resectorization.  The  researchers  selected  a  heavy  traffic  situation  and  shifting  weather  patterns 
as  scenarios  for  this  investigation.  The  approach  was  to  predefine  regions  of  airspace  that  could 
be  allocated  to  one  ARTCC  or  the  other  depending  upon  the  traffic  situation.  This  approach 
represented  a  simple  method  of  dynamic  resectorization  that  could  be  implemented  using  current 
air  traffic  control  equipment. 

A  team  of  human  factors  researchers  and  Subject  Matter  Experts  (SMEs)  conducted  the 
simulation  in  the  Research  Development  and  Human  Factors  Laboratory  at  the  Federal  Aviation 
Administration  William  J.  Hughes  Technical  Center.  We  developed  generic  en  route  airspace 
for  the  study  that  consisted  of  two  adjacent  sectors  from  different  ARTCCs.  We  briefed  every 
controller  on  the  airspace,  sector  configurations  and  the  standard  operating  procedures  for  the 
generic  sectors.  We  coupled  the  briefing  with  hands-on  training  scenarios  to  help  controllers 
quickly  become  familiar  with  the  generic  sectors. 

Twelve  full  performance  level  controllers  participated  in  the  study  over  a  6-week  period.  Each 
week,  two  controllers  arrived  for  3  days  of  simulation  testing.  We  evaluated  controller 
performance  using  objective  measures  produced  by  the  laboratory  simulation  software  and  with 
subjective  measures  provided  by  the  SMEs  using  an  over-the-shoulder  rating  form.  We  assessed 
controller  workload  using  the  National  Aeronautics  and  Space  Administration  Task  Load  Index 
and  the  Air  Traffic  Workload  Input  Technique.  We  measured  ATCS  situation  awareness  using 
self-ratings  on  a  numeric  scale.  In  addition,  controllers  completed  questionnaires  after  each 
scenario  and  at  the  end  of  the  study. 

The  results  indicated  that  dynamic  resectorization  did  not  interfere  with  ATCS  performance. 
Most  of  the  objective  and  subjective  measures  of  performance  indicated  that  there  was  no 
difference  between  fixed  and  dynamic  airspace  boundaries  in  either  of  the  traffic  situations 
examined.  However,  the  results  indicated  slightly  fewer  separation  losses  for  dynamic 
resectorization  in  the  heavy  traffic  scenarios,  although  this  trend  was  not  statistically  reliable. 
There  was  no  difference  in  the  number  of  separation  losses  for  the  shifting  weather  scenarios. 
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The  results  also  indicated  fewer  land  line  communications  for  dynamic  resectorization  in  the 
weather  scenarios.  Dynamically  allocating  a  predefined  area  of  airspace  between  the  sectors 
eliminated  the  need  for  most  of  the  coordination  communications.  In  contrast,  the  heavy  traffic 
scenarios  indicated  slightly  more  land  line  communications  for  dynamic  resectorization.  In  fixed 
boundary  baseline  scenarios,  aircraft  were  simply  handed-off  between  sectors  and  no  land  line 
communications  were  necessary. 

Finally,  the  results  indicated  slightly  lower  NASA-TLX  workload  ratings  in  dynamic 
resectorization  scenarios.  However,  dynamic  resectorization  did  not  reduce  controller  situation 
awareness. 

Future  studies  are  needed  to  explore  different  approaches  to  dynamic  resectorization.  The 
potential  benefits  of  both  horizontal  and  vertical  dynamic  resectorization  need  to  be  examined  as 
well  as  resectorization  between  TRACONs  and  ARTCCs.  The  present  study  identified  specific 
high-density  traffic  and  weather  situations  that  can  benefit  from  dynamic  resectorization. 
However,  dynamic  resectorization  may  not  be  effective  for  all  traffic  situations. 

It  is  important  to  identify  situations  where  dynamic  resectorization  may  be  beneficial  and 
situations  where  resectorization  should  not  be  made.  Other  issues  that  need  investigation  are 
when  airspace  boundaries  should  be  adjusted  and  what  form  they  should  take. 
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1.  Introduction 


Dynamic  resectorization  of  airspace  could  enhance  the  efficient  handling  of  heavy  traffic, 
accommodate  shifting  weather  conditions,  status  changes  in  special  use  airspace,  as  well  as  result 
in  increased  safety  and  in  cost  benefits  to  National  Airspace  System  (NAS)  users.  However, 
depending  upon  how  procedures  are  implemented,  dynamic  resectorization  may  be  disruptive, 
increase  controller  workload,  and  cause  other  negative  consequences.  The  human-in-the-loop 
simulation  described  in  this  report  was  designed  to  investigate  a  specific  implementation  of 
dynamic  resectorization  between  two  adjacent  Air  Route  Traffic  Control  Centers  (ARTCCs). 

The  Human  Factors  FY2000  Program  Baseline  Research  Plan  of  Air  Traffic  Services  assesses 
issues  related  to  airspace  boundary  adjustments.  The  ARA  Performance  Plan  Goal  1  (safety) 
calls  for  the  Federal  Aviation  Administration  (FAA)  to  utilize  baseline  data  to  identify  human 
performance  issues  in  air  traffic  management  (ATM).  This  study  directly  addresses  the  ATS 
Subcommitte  Report  of  the  NAS  ATM  Research  and  Development  Advisory  Committee’s 
mandate  to  focus  research  on  the  ability  of  Air  Traffic  Control  Specialists  (ATCSs)  to  deal  with 
flexible  airspace  (e.g.,  dynamic  resectorization). 

Current  airspace  structure  is  rigid  and  does  not  allow  for  dynamic  resectorization  of  airspace 
boundaries.  Dynamic  resectorization  is  adaptive  and  can  efficiently  handle  heavy  traffic 
situations,  shifting  weather  conditions,  status  changes  in  special  use  airspace,  and  user-preferred 
routes.  Dynamic  resectorization  has  the  potential  to  reduce  aircraft  delays,  decrease  fuel 
consumption,  and  lower  operating  costs  for  the  airline  industry.  The  potential  benefits  for 
ATCSs  are  to  offset  heavy  workload  and  reduce  coordination  and  communications.  However, 
dynamic  resectorization  may  be  disruptive  and  could  have  negative  consequences  for  controller 
situational  awareness  and  performance.  There  are  different  approaches  to  implementing 
dynamic  resectorization  using  current  and  future  automation  tools.  Some  methods  may  be  less 
disruptive  and  more  effective  than  others. 

Inter-facility  dynamic  resectorization  represents  a  radical  change  from  current,  mostly  static 
procedures  that  determine  airspace  boundaries.  If  inter-facility  dynamic  resectorization  is  used 
to  support  increased  flight  flexibility,  it  still  must  provide  controllers  the  cues,  information,  and 
organization  necessary  to  maintain  situation  awareness  and  aircraft  safety.  The  key  is  ensuring 
that  the  dynamic  resectorization  process  itself  does  not  impair  system  efficiency. 

1.1  Background 

Due  to  areas  of  severe  weather,  air  turbulence,  navigational,  or  communications  equipment 
problems,  it  often  becomes  necessary  to  divert  air  traffic  from  their  normal  or  preferred  routes. 
Sometimes,  sectors  become  so  congested  with  traffic  that  aircraft  must  be  diverted  to  avoid  the 
sector.  Allowing  airspace  users  more  flexibility  in  determining  flight  routes  and  the 
implementation  of  Free  Flight  proposals  will  further  exacerbate  these  pressures  over  preferred 
routes  or  sectors  (Planzer  &  Jenny,  1995;  RTCA  1995a,  1995b).  The  increased  flight  flexibility 
associated  with  Free  Flight  could  lead  to  situations  in  which  current  airspace  sector 
configurations  no  longer  match  traffic  flows.  To  accommodate  Free  Flight,  the  sectorization  of 
airspace  will  also  need  to  be  more  flexible,  especially  if  controllers  maintain  responsibility  for 
safe  separation. 
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The  forecast  for  increasing  air  traffic  demands  over  the  next  decade  (Honeywell,  Inc,  1997; 
Wickens,  Mavor,  Parasuraman,  &  McGee,  1998)  will  promote  higher  traffic  density  in  most 
sectors  of  the  NAS.  In  order  to  avoid  heavy  sector  congestion,  the  Traffic  Management  Unit 
(TMU)  may  implement  initiatives  such  as  the  En  Route  Spacing  Program  (ESP),  mile  in-trail 
restrictions,  or  a  ground  delay.  When  these  initiatives  are  set  in  motion,  the  result  is  a  delay  to 
the  aircraft.  Rather  than  implementing  a  ground  delay  or  rerouting  the  aircraft  and  possibly 
causing  heavy  congestion  in  nearby  sectors,  adaptive  airspace  management  techniques  like 
dynamic  resectorization  are  being  proposed  (Eurocontrol,  1998).  Instead  of  rerouting  the  traffic 
to  match  the  existing  resources,  dynamic  resectorization  could  theoretically  optimize  Air  Traffic 
Control  (ATC)  resources  and  lead  to  a  more  balanced  system  workload.  Carlson  and  Rhodes 
(1998)  described  some  adaptive  airspace  management  practices  currently  in  use  in  the 
operational  environment.  Some  involve  the  lateral  reconfiguration  of  sectors  inside  the  same 
facility.  In  other  instances,  ARTCC  and  Department  of  Defense  (DOD)  facilities  delegate  some 
of  their  airspace  to  neighboring  facilities. 

All  cases  except  one1  involve  the  use  of  predefined  airspace  configurations.  Using  Salt  Lake 
City  En  Route  ARTCC  airspace,  researchers  conducted  a  fast-time  simulation  to  investigate  this 
concept  of  dynamic  resectorization.  They  adjusted  sector  boundaries  in  response  to 
representative  traffic  flows  from  the  actual  airspace  (Goldberg  &  Eberlin,  1997;  Honeywell,  Inc., 
1997).  Results  indicated  that  adaptive  sectors  offered  more  user-preferred  routing  thus  reducing 
delays  and  enhancing  aircraft  fuel  efficiency. 

In  another  related  study,  Pawlak,  Bowles,  Goel,  and  Brinton  (1997)  evaluated  the  impact  of 
lateral  resectorization  on  controller  performance.  They  conducted  a  human-in-the-loop 
simulation  in  which  pairs  of  controllers  were  responsible  for  two  adjacent  high-altitude  sectors 
above  Cleveland.  The  authors  assigned  each  controller  to  one  of  the  two  sectors  during  the  five 
40-minute  scenarios  of  the  experiment.  They  designed  the  first  scenario  to  represent  a  baseline 
condition  in  which  the  sector  boundaries  remained  fixed.  The  order  of  presentation  of  the  four 
other  scenarios  was  randomly  determined.  They  defined  these  four  scenarios  by  a  cross  of  two 
independent  variables:  sector  dynamics  and  sector  set.  The  sector  dynamics  varied  in  two  ways. 
In  the  continuously  changing  condition,  they  adjusted  the  sector  boundaries  as  often  as 
necessary.  In  the  interval  condition,  they  optimized  boundaries  only  at  15-minute  intervals. 
There  were  also  two  types  of  sector  set  (limited  and  unlimited).  During  the  limited  set  condition, 
only  seven  possible  new  sector  configurations  could  be  used.  For  the  unlimited  sector  set 
condition,  any  boundary  configuration  was  possible. 

Results  from  this  study  indicated  the  following  recommendations  for  future  research: 

a.  Procedures  for  changing  sector  boundaries  must  be  formalized  to  ensure  that  transitions 
proceed  smoothly. 

b.  Automated  enhancements  can  be  used  to  minimize  the  amount  of  controller  to  controller 
coordination  needed  to  accommodate  a  sector  change. 


'The  Jacksonville  Naval  Air  Station’s  Fleet  Area  Control  and  Surveillance  Facility  (FACSFACJAX)  has  the 
capability  to  dynamically  modify  the  lateral  boundaries  of  restricted  zones,  for  example,  areas  designated  as  off 
limits  due  to  the  presence  of  protected  species  of  whales. 
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c.  The  frequency  with  which  sector  boundaries  can  change  will  be  constrained  by  the 
complexity  of  the  traffic  situation  and  the  complexity  associated  with  making  each 
boundary  change. 

d.  Unless  significant  ATM  system  changes  are  made,  current  radio  frequency  limitations 
and  controller  specialization  in  certain  areas  of  airspace  will  probably  restrict  the 
magnitude  of  boundary  changes. 

e.  New  sector  configurations  may  need  to  be  limited  to  a  pre-defined  set  so  those  controllers 
can  receive  appropriate  training  for  each  configuration. 

f.  Added  flexibility  to  accommodate  weather -systems  or  unusual  traffic  patterns  may  also 
be  beneficial. 

In  an  airspace  with  high  traffic  density,  there  is  higher  probability  for  conflicts  and  increased 
controller  workload.  Airspace  sectors  that  can  be  restructured  to  make  use  of  the  complete 
resources  of  the  ATCSs  have  the  potential  to  increase  overall  system  safety,  provide  a  more 
balanced  workload  for  the  controller,  and  reduce  costly  delays. 

1.2  Purpose 

The  purpose  of  this  study  was  to  conduct  a  human  factors  evaluation  of  the  potential  impact  of 
dynamic  resectorization  between  adjacent  ARTCCs  on  controllers  using  a  real-time  ATC 
simulation.  This  study  compared  operations  between  a  standard  en  route  airspace  with  fixed 
boundaries  to  an  en  route  airspace  with  dynamic  boundaries.  This  should  be  viewed  as  an  initial 
investigation  of  the  dynamic  resectorization  concept  and  not  as  a  comprehensive  assessment. 

1.3  Study  Objective  and  Limitations 

We  examined  the  impact  of  inter-facility  dynamic  resectorization  on  controller  workload, 
communication,  situational  awareness,  control  strategies,  and  performance.  Because  there  are 
current  practices  such  as  airspace  shelving  for  altering  vertical  sector  boundaries,  we  limited  the 
investigation  to  lateral  sector  adjustments. 

2.  Method 


Two  Human  Factors  Specialists  from  the  NAS  Human  Factors  Branch  (ACT-530)  and  two 
ATCS  Subject  Matter  Experts  (SMEs)  conducted  the  simulation  in  the  Research  Development 
and  Human  Factors  Laboratory  (RDHFL)  at  the  FAA  William  J.  Hughes  Technical  Center.  .  A 
team  of  trained  simulation  pilots  operated  aircraft  using  simple  keyboard  commands  and 
communicated  with  the  controllers  using  ATC  phraseology.  Support  engineers  from  ACT-510 
ensured  that  the  simulation  system  functioned  accurately  and  recorded  the  required  performance 
data  properly. 

2.1  Participants 

Current,  non-supervisory,  full  performance  level  ATCSs  participated  in  this  simulation  study. 
We  requested  12  ATCSs  from  at  least  six  different  ARTCCs.  Participants  were  required  to  have 
self-reported  corrected  vision  of  at  least  20/30.  They  ranged  from  3 1  to  56  years  of  age 
(M=44.3)  with  an  average  of  15.4  years  of  ARTCC  experience.  Participants  filled  out  an 
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Informed  Consent  form  explaining  that  their  participation  in  this  study  was  strictly  voluntary  and 
that  their  privacy  was  protected  (Appendix  A).  We  maintained  strict  adherence  to  all  Federal, 
Union,  and  ethical  guidelines  throughout  the  study.  Participants  were  allowed  to  withdraw  from 
the  study  at  any  time  without  penalty.  The  simulation  evaluated  the  concept  of  inter-facility 
dynamic  resectorization  and  not  individual  controllers. 

2.2  Equipment 

The  simulation  equipment  consisted  of  workstations  with  large  high-resolution  displays,  a  voice 
communications  system,  networked  computer  resources,  and  ATCoach  (1996)  simulation 
software. 

As  part  of  the  simulation  materials,  we  printed  and  time  ordered  flight  progress  stripsin  a  strip 
bay  prior  to  the  start  of  each  scenario.  We  audio-video  recorded  the  simulation  and  included  a 
touchscreen  for  the  Air  Traffic  Workload  Input  Technique  (ATWIT)  (Stein,  1985)  in  the  system. 

2.3  Airspace 

The  research  team  selected  the  en  route  environment  and  a  generic  ARTCC  airspace  (Genera 
Center).  A  generic  airspace  has  several  advantages  relative  to  modeling  an  actual  airspace  in 
simulations.  Using  a  generic  airspace,  researchers  can  select  a  cross-section  of  controllers  from 
different  Air  Traffic  facilities  and  quickly  train  them  to  operate  within  the  airspace.  We 
developed  Genera  Center  (ZGN)  using  the  ATCoach  (1996)  simulation  model  that  closely 
replicates  the  en  route  environment.  ZGN  has  the  flexibility  to  interface  with  the  Generic 
Terminal  Radar  Control  (TRACON),  a  generic  airspace  environment  that  was  validated  in  a 
previous  simulation  (Guttman,  Stein,  &  Gromelski,  1995).  ZGN  consists  of  easily  remembered 
fix  names  and  simplified  operating  procedures.  We  divided  ZGN  into  two  separate  center 
configurations  to  simulate  an  inter-facility  operation.  We  gave  an  airspace  briefing  to  each 
participant,  which  described  ZGN  and  pertinent  standard  operating  procedures  (SOPs),  sector 
layouts,  and  jet  routes.  In  this  briefing,  we  also  described  the  areas  of  responsibility  during 
dynamic  resectorization.  This  is  important  because  controllers  would  have  to  be  certified  on 
these  portions  of  adjacent  center  airspace  in  order  for  an  adjustment  to  be  permitted. 

2.4  Traffic  Scenarios 


Controllers  conducted  traffic  in  two  different  experimental  conditions.  In  the  first  condition, 
they  employed  dynamic  resectorization  between  the  two  ARTCC  configurations.  The  second 
condition  involved  current  operating  procedures  for  controlling  and  directing  traffic  between 
ARTCC  facilities  and  served  as  a  baseline  for  comparison.  There  were  four  scenarios  for  each 
condition.  Two  of  the  scenarios  consisted  of  rather  heavy  traffic,  and  two  were  a  combination  of 
medium  traffic  and  a  severe  weather  system.  Each  scenario  was  60  minutes  in  duration  and 
consisted  of  a  mix  of  jet  aircraft  operating  in  instrument  flight  rules  conditions.  All  scenarios 
started  without  any  initial  aircraft  on  the  radar  display.  Then,  aircraft  steadily  appeared,  creating 
a  buildup.  This  level  of  traffic  was  maintained  for  the  duration.  Each  controller  experienced  all 
scenarios  from  each  position  (four  from  one  ARTCC  the  first  day  and  four  from  the  other 
ARTCC  the  following  day).  In  all  scenarios,  controllers  directed  traffic  according  to  current 
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ATC  procedures  (with  the  exception  of  procedural  changes  associated  with  inter-facility 
dynamic  resectorization). 


2.5  Design 

To  evaluate  situations  that  might  have  an  impact  on  the  controller  operating  in  a  dynamic 
airspace,  we  decided  to  limit  our  investigation  to  two  independent  variables:  Airspace  Type  and 
Traffic  Situation.  The  experimental  design  can  be  summarized  as  a  2  X  2  within-subjects  (or 
repeated  measures)  design  with  the  factors  of  Airspace  Type  (fixed  or  dynamic)  and  Traffic 
Situation  (high  density  or  weather).  We  administered  eight  scenarios  in  a  randomized  order 
(Appendix  B).  We  designed  the  scenarios  so  that  the  North  Center  (ZNO)  always  had  the 
problem  (high  density  traffic  or  severe  weather).  The  South  Center  (ZSO)  acquired  an  area  from 
ZNO  through  resectorization.  We  intended  the  resectorization  to  be  a  solution  to  the  traffic 
situation  in  ZNO  without  significantly  impacting  operations  in  ZSO. 

2.5.1  Independent  Variables 

We  examined  these  variables  in  terms  of  two  conditions  over  eight  scenarios: 

a.  High-Density  Traffic  Scenarios 

1 .  Baseline  Fixed  Boundaries  with  High-Density  Traffic  -  This  condition  employed 
current  71 10.65M  ATC  procedures  for  controlling  traffic.  It  consisted  of  a  large 
volume  of  aircraft,  some  of  which  were  transferring  from  ZSO  to  ZNO.  Each 
controller  performed  this  baseline  scenario  from  ZNO  and  ZSO  (see  Figure  la). 

2.  Dynamic  Resectorization  with  High-Density  Traffic  -  This  condition  included  the 
same  traffic  flow  as  in  the  Baseline  High-Density  scenario.  It  started  with  a 
baseline  airspace  configuration.  At  17  minutes  into  the  scenario,  the  airspace  was 
resectorized  as  shown  in  Figure  lb  to  distribute  the  taskload  more  evenly  between 
controllers  and  accommodate  the  large  volume  of  aircaft.  Participants 
experienced  this  scenario  once  from  ZNO  and  once  from  ZSO. 

b.  Weather  Scenarios 

1 .  Baseline  Fixed  Boundaries  with  Weather  -  This  condition  also  employed  current 
71 10.65M  ATC  procedures  for  controlling  traffic.  This  scenario  consisted  of  a 
moderate  volume  of  aircraft  that  is  accompanied  by  severe  weather.  Each 
controller  performed  this  baseline  scenario  from  ZNO  and  ZSO  (see  Figure  2a). 

2.  Dynamic  Resectorization  with  Weather  -  This  condition  included  the  same  traffic 
flow  as  in  the  Baseline  Weather  scenario.  The  scenario  started  with  a  baseline 
airspace  configuration.  At  17  minutes  into  the  scenario,  the  airspace  was 
resectorized  as  shown  in  Figure  2b  to  allow  one  controller  to  have  more  aispace 
available  to  maneuver  aircraft  around  the  weather.  Participants  experienced  this 
scenario  once  from  ZNO  and  once  from  ZSO. 
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Figure  1 .  High-Density  Traffic  Scenarios,  (a)  Airspace  boundaries  for  baseline  scenario,  (b) 
Airspace  boundaries  for  resectorization  scenario  after  resectorization  has  been  completed.  The 
resectorized  portion  of  airspace  in  this  scenario  was  referred  to  as  the  “northeast  corridor.” 


Figure  2.  Weather  Scenarios,  (a)  Airspace  boundaries  for  baseline  scenario,  (b)  Airspace 
boundaries  for  resectorization  scenario  after  resectorization  has  been  completed.  The 
resectorized  portion  of  airspace  in  this  scenario  was  referred  to  as  “thunder  alley.” 

2.5.2  Dependent  Variables 

The  automated  data  collection  system  of  the  RDHFL  produces  a  large  set  of  objective  system 
effectiveness  measures  that  are  typically  examined  in  ATC  simulation  research  (Buckley, 


DeBaryshe,  Hitchner,  &  Kohn,  1983).  Table  1  lists  selected  measures  separated  into  three 
categories:  safety,  capacity,  and  efficiency. 

Table  1.  System  Effectiveness  Measures 


1 -  SAFETY 

NECNF  -  Number  of  Standard  En  Route  Conflicts 

2  -  CAPACITY 

NCOMP  -  Number  of  Flights  Completed 

3  -  EFFICIENCY 

NPTT  -  Number  of  Controller  Push-to-Talk  Communications 

DPTT  -  Cumulative  Duration  of  Controller  Push-to-Talk  Communications 

NALT  -  Frequency  of  Altitude  Changes 

NHDG  -  Frequency  of  Heading  Changes 

NSPD  -  Frequency  of  Airspeed  Changes 

NLL-  Frequency  of  Land  Line  Communications 

DLL  -  Cumulative  Duration  of  Land  Line  Communications 

DIST  -  Distance  Flown  for  All  Flights 


Additionally,  two  SMEs  observed  controllers  for  over-the-shoulder  (OTS)  ratings  of 
performance.  The  SMEs  used  an  observation  form  (Appendix  C)  specifically  designed  for  use  in 
ATC  human  factors  research  (Sollenberger,  Stein,  &  Gromelski,  1997;  Vardaman  &  Stein, 

1998).  Table  2  shows  the  26  different  rating  scales  of  the  observation  form  organized  into  six 
major  performance  categories.  We  sampled  controller  workload  in  real  time  during  each 
scenario  using  the  ATWIT,  a  subjective  rating  method  (Stein,  1985)  and,  upon  completion  of 
each  scenario,  using  the  NASA  Task  Load  Index  (TLX)  subjective  mental  workload  scale  (Hart 
&Staveland,  1988). 

2.6  Procedures 

The  participants  arrived  at  the  RDHFL  in  pairs  for  a  week  of  simulation  testing.  Each  pair 
followed  a  schedule  as  shown  in  Table  3.  Monday  and  Friday  were  scheduled  for  travel. 
Tuesday,  Wednesday,  and  Thursday  consisted  of  project  briefing,  sector  training,  and  simulation 
test  scenarios.  Participants  worked  from  8:00  AM  to  4:30  PM  with  a  lunch  period  and  a  couple 
of  breaks  each  day.  They  filled  out  a  Background  Questionnaire  (Appendix  D),  and  the  research 
team  assigned  a  participant  code.  To  assure  anonymity,  only  the  participant  and  the  research 
team  knew  this  number.  All  questionnaires  and  performance  data  collected  referenced  the 
participant  code  and  not  individual  controller  names. 

After  each  scenario,  participants  completed  a  Post-Scenario  Questionnaire,  the  NASA  TLX,  and 
an  Exit  Questionnaire  (Appendix  D)  on  the  last  day  of  the  study  during  the  final  debriefing. 
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Table  2.  Observation  Rating  Form  (En  Route  Environment) 


I  -  MAINTAINING  SAFE  AND  EFFICIENT  TRAFFIC  FLOW 

1.  Maintaining  Separation  and  Resolving  Potential  Conflicts 

2.  Sequencing  Arrival,  Departure,  and  En  Route  Aircraft  Efficiently 

3.  Using  Control  Instructions  Effectively/Efficiently 

4.  Overall  Safe  and  Efficient  Traffic  Flow  Scale  Rating 

n  -  MAINTAINING  ATTENTION  AND  SITUATIONAL  AWARENESS 

5.  Maintaining  Situational  Awareness 

6.  Ensuring  Positive  Control 

7.  Detecting  Pilot  Deviations  from  Control  Instructions 

8.  Correcting  Errors  in  a  Timely  Manner 

9.  Overall  Attention  and  Situational  Awareness  Scale  Rating 

in  -  PRIORITIZING 

10.  Taking  Actions  in  an  Appropriate  Order  of  Importance 

11.  Preplanning  Control  Actions 

12.  Handling  Control  Tasks  for  Several  Aircraft 

13.  Marking  Flight  Strips  while  Performing  Other  Tasks 

14.  Overall  Prioritizing  Scale  Rating 

IV  -  PROVIDING  CONTROL  INFORMATION 

15.  Providing  Essential  Air  Traffic  Control  Information 

16.  Providing  Additional  Air  Traffic  Control  Information 

17.  Providing  Coordination 

18.  Overall  Providing  Control  Information  Scale  Rating 

V  -  TECHNICAL  KNOWLEDGE 

19.  Showing  Knowledge  of  LOAs  and  SOPs 

20a.  Showing  Knowledge  of  Aircraft  Capabilities  and  Limitations 
20b.  Showing  Effective  Use  of  Equipment 

21.  Overall  Technical  Knowledge  Scale  Rating 

VI  -  COMMUNICATING 

22.  Using  Proper  Phraseology 

23.  Communicating  Clearly  and  Efficiently 

24.  Listening  to  Pilot  Readbacks  and  Requests 

25.  Overall  Communicating  Rating  Scale 


Table  3.  Participant  Schedule 


Tuesday 

|  Wednesday  j 

|  Thursday 

Time 

Activity 

Time 

Time 

Activity 

8:00  - 

9:15 

Project  Briefing 

8:00  - 

9:15 

Test  Scenario  1 

8:00  - 

9:15 

Test  Scenario  5 

9:15  - 

9:45 

Break 

9:15  - 

9:45 

Break 

9:15  - 

9:45 

Break 

9:45  - 

11:00 

Practice  Scenario  1 

9:45  - 

11:00 

Test  Scenario  2 

9:45  - 

11:00 

Test  Scenario  6 

11:00  - 

12:30 

Lunch 

11:00  - 

12:30 

Lunch 

11:00  - 

12:30 

Lunch 

12:30  - 

1:45 

Practice  Scenario  2 

12:30  - 

1:45 

Test  Scenario  3 

12:30  - 

1:45 

Test  Scenario  7 

1:45  - 

2:15 

Break 

1:45  - 

2:15 

Break 

1:45  - 

2:15 

Break 

2:15  - 

3:30 

Debriefing 

2:15  - 

3:30 

Test  Scenario  4 

2:15  - 

3:30 

Test  Scenario  8 

3:30  - 

4:00 

Break 

4:00  - 

4:30 

Final  Debriefing 

Note. 

Practice  scenarios  were  60  minutes  in  duration  with  moderate  traffic. 

Participants  worked  2  practice  scenarios  (one  from  ZNO,  the  other  from  ZSO) 

Test  scenarios  were  60  minutes  in  duration. 

Participants  worked  4  with  high  density  traffic  (2  baseline  and  2  resectorization)  and  4  with  severe  weather  (2  baseline  and  2 
resectorization) 

Post-Scenario  Questionnaires  were  part  of  30-minute  break  period 
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The  SMEs  performed  the  on-the-job-training  subjective  rating  during  each  trial.  They  also 
interacted  with  the  participant  in  the  beginning  and  ending  of  each  trial  and  in  communicating 
with  the  simulation  pilots,  as  needed.  A  voice  communication  link  to  another  room  allowed 
controllers  to  issue  commands  to  the  simulation  pilots. 

We  used  the  ATWIT  (Stein,  1985)  to  assess  controller  workload  during  the  scenario.  The 
ATWIT  provides  an  unobtrusive  and  reliable  means  for  collecting  self-report  ratings  of 
controller  workload  as  they  control  traffic.  A  touchscreen  was  used  to  present  a  workload  rating 
scale  and  record  the  participant  responses.  The  controllers  indicated  their  current  workload  by 
pressing  one  of  the  touchscreen  buttons  labeled  from  1  (low)  to  10  (high).  The  touchscreen  was 
programmed  to  request  controller  input  every  5  minutes  by  emitting  several  beeps  and  presenting 
the  rating  scale.  Participants  had  20  seconds  to  respond.  If  they  did  not  respond  within  that  20 
seconds,  the  maximum  workload  rating  of  10  was  recorded. 

2.7  Training 

We  developed  a  training  program  to  help  controllers  learn  ZGN  and  become  familiar  with  the 
simulation  setup  and  procedures.  A  member  of  the  research  team  described  the  SOPs  of  ZGN 
and  the  resectorization  process  to  controllers  (Appendix  E).  They  then  demonstrated  the  SOPs 
and  resectorization  as  part  of  the  first  practice  scenario.  In  the  remaining  time  scheduled  for 
training,  participants  had  the  opportunity  to  work  an  additional  practice  scenario  in  an  airspace 
with  reconfigurable  sector  boundaries. 

3.  Results 

We  used  Analysis  of  Variance  (ANOVA)  to  determine  the  effects  of  resectorization  on  the 
dependent  measures  collected  in  the  simulation.  ANOVA  is  a  statistical  procedure  for 
determining  whether  the  differences  between  means  are  due  to  the  manipulated  (or  independent) 
variables  or  due  to  chance  alone.  The  results  of  the  analysis  produce  an  F  statistic  and  an 
associated  p  value.  The  p  value  is  the  probability  that  the  differences  in  the  means  are  due  to 
chance  alone.  Researchers  compare  the  p  value  to  a  selected  significance  level  to  determine  if 
the  differences  are  statistically  significant.  By  convention,  a  p  value  that  is  greater  than  .05  is  not 
considered  statistically  significant. 

Researchers  refer  to  the  analysis  associated  with  each  independent  variable  as  a  main  effect  and 
the  analysis  associated  with  the  combinations  of  variables  as  the  interaction  effect.  An 
interaction  occurs  when  the  effects  of  one  variable  are  different  depending  upon  the  level  of 
another  variable.  If  an  interaction  is  significant,  the  experiment  must  be  broken  down  into  its 
basic  components,  referred  to  as  simple  main  effects.  One  simple  main  effect  is  the  difference 
between  Airspace  Type  (fixed  or  dynamic)  and  the  other  is  Traffic  Situation  (high  density  or 
weather).  F  statistics  are  then  computed  for  each  simple  main  effect. 

Significant  main  effects  or  simple  main  effects  with  more  than  two  levels  require  a  post  hoc 
comparison  procedure  in  order  to  determine  which  levels  are  statistically  significant.  In  the 
present  study,  significant  main  effects  for  Traffic  Situation  are  not  very  meaningful  because  the 
weather  and  high-density  scenarios  were  considerably  different  from  each  another.  Rather,  we 
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were  interested  in  main  effects  for  Airspace  Type  and  the  interactions  between  Airspace  Type 
and  Traffic  Situation. 

We  conducted  a  two-way  repeated  measures  ANOVA,  which  was  collapsed  across  both  the 
North  and  South  ARTCCs  for  the  majority  of  the  dependent  measures.  Tables  summarize  the 
results  of  the  analyses  and  report  the  F  statistics  associated  with  the  effects  for  each  dependent 
measure.  Graphs  present  the  means  of  the  experimental  conditions  in  more  detail  for  selected 
dependent  measures. 

3. 1  System  Effectiveness  Measures 

Twelve  separation  losses2  occurred  during  the  96  experimental  runs.  Figure  3  presents  the 
means  of  these  separation  losses  across  experimental  conditions.  There  were  no  differences  in 
separation  losses  in  the  weather  scenarios  as  a  function  of  resectorization.  However,  in  the  high- 
density  traffic  scenario,  participants  had  fewer  separation  losses  when  dynamic  resectorization 
was  employed.  Although  this  difference  was  not  significant  at  the  .05  level,  there  was  a  trend 
toward  significance  [F  (1,11)  =  4.66,  p  =  .0538]. 


Figure  3.  Separation  losses  across  conditions. 

Table  4  shows  the  results  of  the  two-way  ANOVA  for  the  system  effectiveness  measures.  There 
were  several  interactions  between  Airspace  Type  and  Traffic  Situation  for  these  measures.  The 
simple  main  effects  revealed  significant  decreases  in  the  number  of  flights  completed  and  the 
number  of  flights  handled  in  the  resectorization  condition.  Table  5  shows  the  results  of  the 
analysis  of  simple  main  effects.  As  expected,  there  were  significant  differences  in  the  high- 
density  scenario  for  flights  completed  and  flights  handled  as  a  function  of  Airspace  Type. 


2  A  loss  of  separation  occurs  when  aircraft  do  not  have  either  5  nm  lateral  separation  or  2,000  ft  vertical  separation 
when  above  29,000  ft  or  1 ,000  ft  vertical  separation  when  below  29,000  ft. 
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Table  4.  F  Statistics  Obtained  From  the  Two-Way  ANOVA  Performed  on  the  System 

Effectiveness  Measures 


Measure 

Main  Effect:  Airspace 

Main  Effect:  Traffic 

Interaction  Effect 

NECNF  -  standard  conflicts 

F  (1,11)  =  4.66  f 

F  (1, 11)  =  0.13 

F  (1,11)  =0.80 

NCOMP  -  flights  completed 

F  (1,11)  =84.57* 

F  (1,  11)  =  59.60* 

F  (1,11)  =79.26* 

NPTT-  number  of  transmissions 

F  (1, 11)  =  4.37 

F  (1, 11)  =  5.76* 

F  (1, 11)  =  3.41 

DPTT  -  duration  of  transmissions 

F  (1, 11)  =  0.09 

F  (1, 11)  =  1.09 

F  (1, 11)  =  8.25* 

NALT  -  altitude  changes 

F  (1, 11)  =  0.13 

F  (1, 11)  =  12.23** 

F  (1, 11)  =  0.13 

NHDG  -  heading  changes 

F  (1, 11)  =  0.12 

F  (1, 11)  =  11.34** 

F  (1, 11)  =  0.23 

NSPD  -  speed  changes 

F  (1, 11)  =  0.08 

✓ — \ 

II 

to 

to 

F  (1, 11)  =  0.35 

NLL  -  number  of  land  line  comms. 

F  (1, 11)  =  35.57* 

F  (1,  11)=  167.28* 

F  (1,11)  =  196.95* 

DLL  -  duration  of  land  line  comms. 

F  (1, 11)  =  2.56 

F  (1, 11)  =  64.30* 

F  (1,11)  =  14.11* 

DIST  -  distance  of  flights 

F  (1, 11)  =  0.19 

F  (1,  11)  =  24.98* 

F  (1, 11)  =  0.59 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 

**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p<  .01 

f  indicates  an  effect  that  was  not  statistically  significant  but  nearly  significant  with  ap  <  .06 


Table  5.  Mean  Completed  Flights  and  F  Statistics  Obtained  from  the  Analysis  of  Simple  Main 

Effects 


High  Density 

Weather 

Baseline 

Dynamic 

F  Statistics 

Baseline 

Dynamic 

F  Statistics 

48.87 

43.50 

176.86** 

43.33 

43.54 

0.23 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 
**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .01 


The  efficiency  indicators  showed  several  interactions  between  Airspace  Type  and  Traffic 
Situation.  There  were  no  significant  differences  found  for  NPTT  as  a  function  of  Airspace  Type, 
however,  there  was  a  significant  interaction  obtained  for  the  duration  of  those  communications. 
Figure  4  represents  the  Airspace  Type  by  Traffic  Situation  interaction  of  the  mean  total  duration 
of  push-to-talk  transmissions  (DPTT).  Table  6  shows  the  results  of  the  analysis  of  simple  main 
effects.  In  the  high-density  scenario,  the  duration  of  controller  transmissions  was  not  as  brief  in 
the  baseline  condition  as  compared  to  when  they  resectorized.  Under  the  weather  scenario,  it 
appears  to  turn  in  the  opposite  direction.  However,  this  difference  was  not  significant. 
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Figure  4  Airspace  type  by  traffic  situation  interaction  for  duration  of  push-to-talk  transmissions. 


Table  6.  Mean  DPTT  and  F  Statistics  Obtained  from  the  Analysis  of  Simple  Main  Effects 


High  Density 

|  Weather 

Baseline 

565.04 

Dynamic 

536.88 

F  Statistics 
8.57* 

Baseline 

551.42 

Dynamic 

572.63 

F  Statistics 

L41 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 
**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .01 


Two  other  efficiency  indicators,  frequency  (NLL)  and  duration  of  land  line  (DLL) 
communications,  showed  significant  interactions.  Figure  5  and  Figure  6  illustrate  these 
relationships. 

Controllers  utilized  the  land  line  for  coordination  of  traffic  between  the  ZNO  and  ZSO.  Analysis 
of  simple  main  effects  for  these  interactions  revealed  that  for  the  high-density  traffic  situation, 
dynamic  resectorization  required  more  land-line  communications  than  the  baseline.  For  the 
weather  situation,  the  reverse  was  true.  The  baseline  scenario  required  considerably  more 
coordination  than  the  dynamic  resectorization  scenario.  The  duration  of  ground  to  air 
communications  under  resectorization  went  up  during  the  weather  scenario,  but  land-line 
coordination  calls  were  significantly  reduced.  Table  7  and  Table  8  show  the  results  of  the 
analysis  of  simple  main  effects. 
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Figure  5.  Airspace  type  by  traffic  situation  interaction  for  number  of  land  line  communications. 
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Figure  6.  Airspace  type  by  traffic  situation  interaction  for  duration  of  land  line  communications. 
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Table  7.  Mean  NTT,  and  F  Statistics  Obtained  from  the  Analysis  of  Simple  Main  Effects 


High  Density 

Weather 

Baseline 

10.33 

Dynamic 

13.96 

F  Statistics 
15.01** 

Baseline 

45.88 

Dynamic 

25.79 

F  Statistics 
92.53** 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 
**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  g  <  .01 


Table  8.  Mean  DLL  and  F  Statistics  Obtained  from  the  Analysis  of  Simple  Main  Effects 


High  Density  ! 

Weather 

Baseline 

16.25 

Dynamic 

23.83 

F  Statistics 
5.99* 

Baseline 

81.54 

Dynamic 

54.08 

F  Statistics 
6.79* 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 
**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .01 


3.2  SME  Ratings 

Table  9  shows  the  results  of  the  two-way  ANOVA  for  the  observer  ratings.  The  F  statistics 
indicate  that  Airspace  Type  had  a  strong  effect  on  almost  half  of  the  observer  ratings.  For  these 
significant  differences,  ratings  were  lower  in  the  baseline  scenarios.  There  was  a  significant 
interaction  between  Airspace  Type  and  Traffic  Situation  for  preplanning  control  actions.  A 
simple  main  effects  analysis  of  this  relationship  revealed  that  observers  rated  participants 
significantly  higher  in  the  high-density  traffic  situation  when  dynamic  resect orizati on  was 
implemented.  Table  10  shows  the  results  of  the  analysis  of  simple  main  effects.  Figure  7 
illustrates  this  interaction. 

3.3  Controller  Ratings 

3.3.1  NASA  Task  Load  Index 


We  computed  an  unweighted  total  subjective  workload  score  with  a  range  of  zero  to  120  for  each 
participant  by  summing  the  responses  on  the  six  subscales  of  the  NASA-TLX.  A  two-way 
ANOVA  performed  on  these  scores  revealed  a  significant  main  effect  for  Airspace  Type 
[F(l,l  1)  =  38.77].  Participants  rated  both  scenarios  as  more  workload  intensive  when  they  were 
controlling  traffic  in  the  baseline  airspace  configuration.  This  suggests  that  they  perceived  a 
positive  impact  as  a  function  of  resectorization  on  their  workload  when  they  thought  about  it 
after  the  runs.  However,  there  was  no  significant  interaction  between  Airspace  Type  and  Traffic 
Situation  for  these  workload  scores.  We  present  the  mean  TLX  scores  in  Figure  8. 
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Table  9.  F  Statistics  Obtained  From  the  Two-Way  ANOVA  Performed  on  the  Observer  Ratings 


Rating 

Main  Effect:  Airspace 

Main  Effect:  Traffic 

Interaction  Effect 

L  Maintaining  separation 

F(l,  11)  =  0.82 

F(l,  11)  =  0.27 

F(l,  11)  =  3.80. 

2.  Sequencing  traffic 

F(  1,  11)  =  4.94* 

F(l,  11)  =  5.43* 

F(l,  11)  =  2.27 

3.  Using  control  instructions 

F  (1,  11)  =  2.47 

F(l,  11)  =  6.32* 

F(l,  11)  =  0.14 

4.  Overall  traffic  flow 

F  (1, 11)  =  3.46 

F(l,  11)  =  3.02 

F(l,  11)=  1.21 

5.  Maintaining  awareness 

F(l,  11)  =  4.14 

F(l,  11)  =  2.78 

F(l,  11)  =  0.33 

6.  Ensuring  positive  control 

F  (1,  11)  =  11.99** 

F(l,  11)=  1.18. 

F(l,  11)  =  0.07 

7.  Detecting  pilot  deviations 

F(l,  11)=  12.00** 

F(l,  11)  =  6.26* 

F(l,  11)  =1.68 

8.  Correcting  own  errors 

F  (1, 11)  =  5.18* 

F(l,  11)=  1.73 

F(l,  11)  =  0.00. 

9.  Overall  attention  &  awareness 

F  (1,  11)  =  8.19* 

F(l,  11)  =  4.87* 

F(l,  11)  =  0.12 

10.  Taking  action  in  order 

F(l,  11)  =  2.32 

F(l,  11)  =  0.03 

F(l,  11)  =  0.17 

11.  Preplanning  control  actions 

F(l,  11)  =  3.67 

F(l,  1 1)  =  4.12 

F(l,  11)  =  7.71* 

12.  Handling  control  tasks 

F(l,  11)  =  8.37* 

F(l,  11)  =  0.30 

F(l,  11)  =  0.19 

13.  Marking  flight  strips 

F(l,  11)  =  5.83* 

F  (1,  11)  =  25.91** 

F(l,  11)  =  0.56 

14.  Overall  prioritizing 

F  (1, 11)=  10.53** 

F(l,  11)=  10.91** 

F(l,  11)  =  0.03 

15.  Providing  essential  info 

F  (1, 11)  =  6.40* 

F(l,  11)  =  0.85 

F(l,  11)  =  0.02 

16.  Providing  additional  info 

F(l,  11)  =  6.37* 

F(l,  11)  =  5.58* 

F(l,  11)  =  0.51 

17.  Providing  coordination 

F  (1,  11)  =  3.28 

F(l,  11)=  1.91 

F(l,  11)  =  0.01 

18.  Overall  providing  info 

F  (1, 11)  =  5.31* 

F  (1,  11)  =  3.63 

F(l,  11)  =  0.00 

19.  Knowing  LOAs  and  SOPs 

F  (1, 11)  =  2.43 

F(l,  11)  =  0.09 

F(l,  11)  =  0.00 

20a.  Knowing  aircraft  capabilities 

F(l,  11)  =  0.56 

F(l,  11)  =  2.05 

F(l,  11)  =  0.01 

20b.  Effective  use  of  equipment 

F  (1, 11)=  14.57** 

F(l,  11)  =  0.74 

F(l,  11)  =  0.93 

21.  Overall  technical  knowledge 

F  (1, 11)=  11.56** 

F(l,  11)=  1.67 

F(l,  11)  =  0.05 

22.  Using  proper  phraseology 

F  (1,  11)  =  0.63 

F  (1,  11)  =  2.16. 

F(l,  11)  =  0.70 

23.  Communicating  clearly 

F(l,  11)  =  0.31 

F(l,  11)  =  0.01 

F(l,  11)  =  2.19 

24.  Listening  to  pilots 

F(l,  11)  =  0.04 

F(l,  11)  =  0.08 

F(l,  11)  =  0.75 

25.  Overall  communicating 

F  (1,  11)  =  0.01 

F  (1,  11)  =  0.20 

F(l,  11)  =  0.50 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 
**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .01 


Table  10.  Mean  OTS  SME  Ratings  of  Preplanning  Control  Actions  and  F  Statistics  Obtained 

from  the  Analysis  of  Simple  Main  Effects 


High  Density  i 

Weather 

Baseline 

5.46 

Dynamic 

6.25 

F  Statistics 

14  44** 

Baseline 

5.42 

Dynamic 

5.25 

F  Statistics 
0.40 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 
**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  g  <  .01 
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Figure  7.  Airspace  type  by  traffic  situation  interaction  for  preplanning  control  actions. 


Figure  8.  Mean  NASA  TLX  scores. 
3.3.2  ATWIT 


In  contrast  to  the  TLX,  ATWIT  reflects  workload  estimates  in  real  time.  Table  1 1  shows  the 
results  of  the  two-way  ANOVA  for  both  the  TLX  scores  and  the  ATWIT  ratings.  Figure  9 
illustrates  the  ATWIT  ratings  as  a  function  of  Airspace  Type  and  Traffic  Situation.  A  two-way 
ANOVA  revealed  a  significant  interaction  between  these  variables.  Table  12  shows  the  results 
of  the  analysis  of  simple  main  effects.  The  F  statistics  indicate  a  significant  decrease  in 
controller  workload  for  the  weather  scenario  when  dynamic  resectorization  was  employed. 
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Table  11.  F  Statistics  Obtained  From  the  Two-Way  ANOVA  Performed  on  NASA  TLX  and 

ATWIT  Workload  Ratings 


Measure _ Main  Effect:  Airspace  Main  Effect:  Traffic  Interaction  Effect 

NASA  TLX  F(l,  11)  =  38.77**  F(l,  11)  =  39.04**  F(l,  11)  =  1.76 

ATWIT _ _ _ F(l,  11)  =  5.54* _ F(l,  11)=  15.81**  F(l,  11)  =  5.51* 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 
**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  g  <  .01 
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Figure  9.  Airspace  type  by  traffic  situation  interaction  for  atwit  ratings. 


Table  12.  Mean  ATWIT  Ratings,  F  Statistics  Obtained  from  the  Analysis  of  Simple  Main 

Effects 


High  Density  1 

Weather 

Baseline 

5.14 

Dynamic 

5.03 

F  Statistics 
.33 

Baseline 

5.90 

Dynamic 

5.22 

F  Statistics 
9.89** 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 
**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  g  <  .01 


For  the  high-density  traffic  situation  when  dynamic  resectorization  occurred,  there  was  a  slight, 
though  non-significant,  decrease  in  controller  workload.  In  real  time,  differences  still  existed, 
but  they  were  not  quite  as  clear.  To  better  understand  these  differences,  the  mean  ratings  were 
broken  down  over  time  by  scenario  and  Airspace  Type  and  are  depicted  in  Figures  10  and  1 1. 
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Figure  10.  ATWIT  interval  data  for  the  high  density  traffic  scenario. 
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Figure  11.  ATWIT  interval  data  for  the  weather  scenario. 
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3.3.3  Post-Scenario  Questionnaire  Ratings 


Table  13  shows  the  results  of  the  two-way  ANOVA  for  controller  ratings.  The  F  statistics 
indicate  that  participants  felt  they  performed  better  and  rated  their  overall  situational  awareness 
higher  when  they  were  working  traffic  in  dynamic  resectorization  scenarios  regardless  of  traffic 
situation.  There  were  no  significant  interactions  between  Airspace  Type  and  Traffic  Situation 
for  these  two  measures.  Figure  12  and  Figure  13  illustrate  these  main  effects. 


Table  13.  F  Statistics  Obtained  from  the  Two-Way  ANOVA  Performed  on  the  Post-Scenario 

Questionnaire  Ratings 


Rating 

Main  Effect:  Airspace 

Main  Effect:  Traffic 

Interaction  Effect 

1 .  How  coordination  was  affected 

2.  Scenaro  difficulty 

3.  Performance 

4.  Overall  situational  awareness 

5.  Overall  physical/mental  workload 

6.  Traffic  flow  realism 

7.  Simulation  pilot  performance 

F(l,  11)  =  3.34 

F(  1, 11)=  13.19** 

F  (1, 11)=  12.20** 

F  (1,  11)  =  5.68* 
F(l,  11)  =  2.94 

F(l,  11)  =  2.58 

F(l,  11)  =  0.08 

F(l,  11)=  18.54** 

F  (1, 11)  =  21.33** 
F(l,  11)  =  2.69 

F(l,  11)  =  1.17 

F(l,  11)  =  22.00** 
F(l,  11)=  1.20 

F(l,  11)=  1.07 

F(l,  11)  =  11.65** 
F(l,  11)  =  5.65 

F(l,  11)  =  3.81 

F(l,  11)  =  0.09 

F(l,  11)  =  0.00 

F(l,  11)  =  3.12 

F  (1, 11)  =  0.48 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 
**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .01 
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Figure  12.  Post-Scenario  Questionnaire  ratings  of  performance. 
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Figure  13.  Post-Scenario  Questionnaire  ratings  of  situational  awareness. 


This  two-way  ANOVA  did  reveal  significant  interactions  for  ratings  on  coordination  and 
scenario  difficulty  as  a  function  of  Airspace  Type  and  Traffic  Situation.  Table  14  and  Table  15 
show  the  results  of  the  analysis  of  simple  main  effects.  The  F  statistics  indicate  that  baseline 
airspace  configuration  had  more  of  an  impact  on  participant  coordination  but  only  for  the 
weather  traffic  situation  [F  (1,1 1)  =  13.82],  For  scenario  difficulty,  the  F  statistics  revealed  that 
controllers  rated  the  weather  scenario  in  the  baseline  airspace  configuration  as  significantly  more 
difficult  than  the  same  weather  scenario  in  the  dynamic  airspace  configuration.  Figure  14  and 
Figure  15  illustrate  these  two  interactions. 


Table  14.  Mean  Post-Scenario  Ratings  of  Coordination  and  F  Statistics  Obtained  from  the 

Analysis  of  Simple  Main  Effects 


High  Density  1 

Weather 

Baseline 

4.83 

4  n  ntn 

Dynamic 

4.96 

F  Statistics 
0.08 

Baseline 

7.00 

Dynamic 

5.62 

F  Statistics 
13.82** 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 
**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .01 


Table  15.  Mean  Post-Scenario  Ratings  of  Scenario  Difficulty  and  F  Statistics  Obtained  from  the 

Analysis  of  Simple  Main  Effects 


High  Density 

Weather 

Baseline 

6.13 

4  in rli r'ntrif,  n 

Dynamic 

5.67 

F  Statistics 
1.19 

C _ *  „  t  _  ' _ 1ST. _ 

Baseline 

7.67 

_  1  r  r\ 

Dynamic 

6.13 

F  Statistics 
26.65** 

*  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .05 
**  indicates  a  statistically  reliable  effect  at  a  significance  level  of  p  <  .01 
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Figure  14.  Airspace  type  by  traffic  situation  interaction  for  overall  coordination. 


Figure  15.  Airspace  type  by  traffic  situation  interaction  for  difficulty. 

The  Post-Scenario  Questionnaire  Items  9,  10,  11,  and  12  were  to  be  filled  out  only  if  the 
participant  had  just  completed  a  scenario  in  which  dynamic  resectorization  had  occurred.  We 
assumed  that  during  the  period  of  the  scenario  in  which  a  boundary  adjustment  was  employed, 
there  was  a  potential  for  confusion.  Those  four  questions  focused  on  that  important  transition 
time.  Table  16  shows  the  controller  responses  to  this  portion  of  the  Post-Scenario  Questionnaire. 
Combining  the  ratings  from  both  ZNO  and  ZSO  does  not  accurately  show  the  impact  on  human 
performance,  therefore  the  ratings  have  been  broken  down  by  ARTCC  and  also  by  scenario.  We 
conducted  multiple  dependent  f-tests  on  these  ratings  and  found  no  questions  to  differ  as  a 
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Table  16.  Post-Scenario  Questionnaire  Resectorization  Ratings 


Question 

Scale  Anchors 

Grand 

Mean 

(SD) 

Scenario 

ZNO 

Mean  (SD) 

zso 

Mean  (SD) 

/-value 

9.  During  the  time  of  the 
boundary  adjustment, 
what  was  the  effect  on 
your  performance? 

(1) 

Extremely 

negative 

(10) 

Extremely 

positive 

6.65  (2.06) 

High  Density 

Weather 

7.00(2.13) 

7.00  (2.13) 

6.16(1.85) 

6.41  (2.23) 

<  (11)  =  0.99 

r  (11)=  1.13 

10.  During  the  time  of  the 
boundary  adjustment, 
what  was  the  effect  on 
your  physical  and 
mental  workload? 

(1) 

Extremely 

negative 

(10) 

Extremely 

positive 

6.21  (1.99) 

High  Density 

Weather 

6.67  (2.38) 

6.67  (1.96) 

5.58  (1.24) 

5.91  (2.23) 

<  (11)  =1.78 

<  (11)  =  1.43 

1 1 .  During  the  time  of  the 
boundary  adjustment, 

how  difficult  was  it  to 
maintain  your 
situational  awareness? 

CD 

Not  very 
difficult 

(10) 

Extremely 

difficult 

3.00(1.75) 

High  Density 

Weather 

2.66(1.55) 

3.41  (1.67) 

3.25  (2.30) 

2.66(1.43) 

<(H)  =  0.21 

/  (11)  =  1.56 

12.  During  the  time  of  the 
boundary  adjustment,  to 
what  extent  was  your 
coordination  affected? 

(1) 

Very 

little 

(10) 

A  great 
deal 

4.37  (2.41) 

High  Density 

Weather 

3.75  (2.63) 

5.83  (2.58) 

3.75  (2.63) 

3.83  (2.08) 

<(11)  =  -.72 

<  (11)=  1.97 

function  of  what  ARTCC  and/or  scenario  the  participants  were  working.  Overall,  however, 
controllers  rated  that  the  dynamic  resectorization  process  had  only  a  positive  effect  on  their 
performance  and  workload.  For  the  most  part,  they  felt  that  it  was  not  very  difficult  to  maintain 
their  situational  awareness  and  that  the  process  had  little  or  no  impact  on  their  coordination. 

3.3.4  Exit  Questionnaire 


Table  17  shows  the  controller  responses  to  questions  in  the  exit  questionnaire.  As  shown, 
controllers  found  the  simulation  to  be  realistic  and  the  dynamic  resectorization  concept  to  be 
feasible  within  the  NAS.  Controllers  also  indicated  that  the  ATWIT  procedure  did  not  interfere 
with  their  performance.  In  addition,  they  felt  a  positive  impact  on  their  performance  when 
dynamic  resectorization  was  employed  regardless  of  which  ARTCC  they  were  working. 
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Table  17.  Exit  Questionnaire  Ratings 


In  general,  how  realistic  was  the  simulation? 


To  what  extent  did  the  ATWIT  probe  technique 
interfere  with  your  performance? 

While  you  were  controlling  traffic  in  ZNO,  what 
type  of  impact  did  the  boundary  adjustment 
have  on  your  performance? 


(1) 

Not  very 
realistic 

(1) 

Not  very  much 

(1) 

Very 
negative 


|4  While  you  were  controlling  traffic  in  ZSO,  what 
type  of  impact  did  the  boundary  adjustment 
have  on  your  performance? 

i.  Circle  the  number  that  best  describes  the 
feasibility  of  inter-facility  boundary 
adjustment  in  the  National  Airspace  System. 


(1) 

Very 

negative 

(1) 

Not  very 
feasible 


(10) 

Extremely 

realistic 

(10) 

A  great  deal 

(10) 
Very 
positive 


(10) 

Very 

positive 

(10) 

Extremely 

feasible 


1.91  (1.37) 


7.75  (2.22) 


6.75  (2.34) 


7.08  (3.02) 


4  Discussion  and  Conclusions. 

The  present  study  investigated  ^ 

workload,  and  situation  awareness.  Our  aPPro  between  two  adjacent  ARTCCs.  In  both 

the  adjacent  ZSO  was  the  solution. 

Airspace  Type  did  have  an  effect  on  the  system; 

few  separation  losses  overall, ^contra '  ers f  tiQn  It  was  not  statistically  significant,  but 
situation  operating  in  the  baseline  airsp  g  ion  losses  in  the  high-density  traffic 

^Td^e"cPreCCEion  configuration  Controllers  had  the  same  amount  of 

separation  losses  during  the  weather  situation  regardless  of  Airspace  yp  . 

On  the  surface,  given  the  high  quality  of  the  concern  "" 

(96),  the  overall  number  of  errors  does  ^  (w0  {veryPlow  on  the  100-pt  scale). 

SSESK  concern.  Controllers  reported  that  the  simulation  was 


3  Aircraft  Proximity  Index  (API)  -  a  weighted  measure 
a  minor  violation  of  the  separation  standards. 


of  conflict  intensity  where  100  is  a  mid-air  collision  and 


1 


is 
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realistic.  We  asked  the  controllers  to  work  in  novel  conditions,  and  this  could  have  increased  the 
error  rate  somewhat. 

In  general,  capacity  remained  constant  because  controllers  did  not  hold  traffic.  However,  there 
were  differences  in  efficiency  indicators.  Significant  interactions  of  Airspace  Type  by  Traffic 
Situation  were  obtained  for  frequency  and  duration  of  landline  communications  (NT  I.  and  DLL, 
respectively).  These  results  for  the  weather  scenario  are  rather  clear.  In  the  baseline  airspace 
configuration,  the  ZNO  controller  had  to  point  out  each  aircraft  that  was  deviating  around  the 
thunderstorms.  Each  point-out  required  a  land  line  communication  with  ZSO.  This  coordination 
was  eliminated  in  the  dynamic  resectorization  configuration  because  the  ZNO  controller 
acquired  a  portion  of  ZSO,  thus  reducing  land  line  communication. 

The  high-density  scenario  played  out  somewhat  differently.  There  was  a  significant  increase  in 
NLL  and  DLL  in  the  dynamic  resectorization  configuration  for  this  traffic  situation.  Overall, 
there  were  few  land  line  communications  during  the  high-density  scenario  regardless  of  Airspace 
Type.  However,  when  dynamic  resectorization  was  called  for,  there  were  between  three  and  five 
(depending  upon  individual  controller  style)  aircraft  in  the  northeast  corridor.  As  part  of  the 
dynamic  resectorization  procedures,  aircraft  within  the  portion  of  airspace  that  was  being 
resectorized  required  coordination.  This  resulted  in  small,  yet  significant,  increases  in  both 
frequency  and  duration  of  land-line  communications  for  the  high-density  traffic  situation  in  the 
dynamic  resectorization  configuration.  The  relationship  between  sector  structure, 
resectorization,  and  communication  is  complicated. 

The  observer  ratings  of  controller  performance  varied  as  a  function  of  Airspace  Type  for  about 
half  of  the  different  rating  scales  included  in  the  OTS  rating  form.  For  these  significant 
differences,  ratings  were  lower  when  participants  were  controlling  traffic  in  the  baseline  airspace 
configuration.  The  SMEs  were  involved  in  the  simulation  from  the  initial  stages  of  experimental 
design  to  the  airspace  and  scenario  construction.  Therefore,  one  may  suggest  some  sort  of 
observer  bias  in  these  participant  performance  ratings.  However,  the  subjective  observer  ratings 
were  consistent  with  the  objective  system  effectiveness  measures  and  both  indicated  that 
dynamic  resectorization  did  not  interfere  with  controller  performance.  Furthermore,  the  observer 
ratings  were  consistent  with  the  participant’s  self-ratings  of  workload,  performance,  and  situation 
awareness. 

Participants  and  observers  saw  resectorization  as  positive.  They  expressed  the  belief  that  it 
reduced  their  workload.  Participant  perception  did  vary  somewhat  from  real  time  ATWIT  to 
post  hoc  TLX  metrics.  When  they  had  time  to  think  about  the  impact  of  resectorization,  their 
views  were  somewhat  more  positive  then  when  they  were  still  working  traffic.  The  TLX  data 
revealed  significant  decreases  in  subjective  workload  while  operating  in  the  dynamic 
resectorization  configuration  regardless  of  scenario  type,  but  the  ATWIT  data  showed  little 
movement.  An  examination  of  the  mean  ATWIT  ratings  over  time  revealed  that,  in  the  high 
density  scenario  under  baseline  airspace  conditions,  the  workload  ratings  in  ZNO  climbed 
sharply  during  the  first  15  minutes  of  the  scenario  where  they  remained  for  the  duration.  The 
ZSO  ratings  rose  initially  in  the  first  10  minutes  and  plateaued  at  a  lower  level  until  near  the  end 
of  the  scenario  where  they  began  to  drop  somewhat.  During  the  same  traffic  situation  under  the 
resectorized  airspace,  the  ZNO  ratings  over  time  were  significantly  less,  but  the  interesting 
difference  for  ZSO  was  the  noticeable  spike  in  the  ATWIT  ratings  for  the  prompt  following  the 
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acquisition  of  the  northeast  corridor.  Initially,  controllers  in  ZSO  felt  that  gaining  the  northeast 
corridor  added  to  their  workload.  However,  their  ratings  then  dropped  and  closely  mirrored  the 
ZNO  ratings  for  the  duration  of  the  scenario.  Although  the  differences  in  the  high-density 
scenario  between  airspace  types  were  in  the  right  direction,  they  were  not  significant.  These 
results  could  possibly  be  attributed  to  either  the  operational  transition  or  the  scenario  design. 

We  created  optimal  situations  for  dynamic  resectorization.  Ideally,  dynamic  resectorization 
would  be  employed  between  facilities  when  one  ARTCC  is  predicted  to  exceed  capacity  while 
an  adjacent  ARTCC  is  operating  well  below  capacity.  This  particular  scenario  is  difficult  to 
examine  in  a  simulation.  One  way  to  examine  this  may  be  to  create  a  baseline  situation  in  which 
one  participant  would  be  struggling  with  heavy  traffic  flow  while  the  other  would  be  bored  with 
light  traffic  flow.  In  order  to  avoid  this  potential  problem,  we  had  the  ZSO  controller  working  a 
moderate  amount  of  traffic  while  the  ZNO  controller  had  a  high  traffic  count.  Past  simulation 
experience  suggests  that  it  is  not  generally  in  anyone’s  interests  to  put  a  controller  potentially 
over  the  line  of  his/her  capabilities.  The  current  study  was  basically  a  compromise  design,  which 
likely  mirrors  some  but  not  all  real  world  conditions. 

In  the  weather  scenario,  there  was  a  small  yet  significant  decrease  in  real-time  workload  ratings 
in  the  dynamic  resectorization  configuration.  The  mean  ratings  over  time  indicate  that  the 
allocation  of  thunder  alley  from  ZSO  to  ZNO  resulted  in  a  subsequent  decrease  in  ratings 
obtained  from  ZSO  while  the  ZNO  ratings  remained  fairly  consistent  with  the  ratings  obtained 
from  the  same  traffic  situation  when  resectorization  did  not  take  place.  While  the  research  team 
designed  the  traffic  problem  for  the  ZNO,  the  baseline  ATWIT  ratings  over  time  indicate  that  the 
weather  scenario  was  also  somewhat  of  a  problem  for  the  ZSO.  Resectorization  appears  to  have 
solved  this  problem,  at  least  in  terms  of  workload  ratings  for  ZSO.  However,  ZNO  workload 
ratings  do  not  appear  to  differ  as  a  function  of  resectorization. 

Overall,  the  objective  and  subjective  data  collected  during  this  experiment  support  the  fact  that 
resectorization  did  not  interfere  with  performance.  In  addition,  the  post-scenario  estimates  of 
workload  declined  in  the  scenarios  in  which  dynamic  resectorization  was  implemented.  Most 
importantly,  the  results  from  this  study  indicated  that  if  resectorization  is  accomplished  in  a 
timely  manner,  it  does  not  negatively  impact  the  controller  whatsoever.  Of  course,  the  research 
team  in  the  present  study  investigated  a  specific  type  of  resectorization  using  predefined  regions 
of  airspace  in  conditions  that  were  designed  to  be  optimal  for  resectorization  to  take  place. 

However,  several  studies  are  needed  to  explore  different  approaches  to  dynamic  resectorization. 
In  addition,  there  are  a  number  of  operational  and  technical  questions  that  need  to  be  addressed. 
In  particular,  when  should  you  resectorize?  The  goal  of  investigating  this  question  is  to 
determine  the  optimum  point  for  resectorization  to  occur.  Too  early,  is  an  inefficient  use  of 
resources,  while  changing  airspace  after  the  controller(s)  is/are  already  busy  may  have  negative 
consequences  in  terms  of  his/her  ability  to  safely  and  efficiently  control  traffic.  A  predictive 
capability,  such  as  a  measure  of  airspace  complexity  (i.e.,  dynamic  density),  would  not  only  have 
to  account  for  the  sector  complexity  at  some  predetermined  look-ahead  time  but  would  also  be 
required  to  incorporate  the  transition  time  of  the  system  to  accommodate  the  resectorization 
process.  For  example,  if  a  dynamic  density  index  indicated  that  the  sector  complexity  was  going 
to  exceed  a  given  threshold  in  xx  minutes  and  the  transition  period  for  the  resectorization  process 
is  30  minutes,  then  the  controller  working  this  sector  should  receive  notification  30  +xx  minutes 
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in  advance.  Is  this  amount  of  time  sufficient  for  the  controller  to  work  in  and  become 
comfortable  with  this  new  airspace  configuration  before  the  predicted  rush?  This  amount  of  time 
may  be  more  than  adequate,  or  controllers  may  require  considerably  more  time  to  become 
accustomed.  It  is  this  period  of  time  that  needs  to  be  addressed  in  simulation. 

Another  question  includes  how  often  can  you  resectorize?  Requiring  a  controller  to  work 
multiple  airspace  configurations  during  his/her  shift  is  probably  not  feasible.  There  is  no  value 
added  in  changing  airspace  boundaries  to  accommodate  a  traffic  push  that  is  projected  to  last  for 
only  a  short  time  or  to  re-route  traffic  around  a  small  weather  cell.  In  terms  of  safety  and 
efficiency,  what  is  the  minimum  period  of  time  a  traffic  push  or  weather  cell  has  to  be  present  in 
order  to  warrant  a  resectorization,  considering  not  only  the  limitations  of  automation  but  also 
without  compromising  the  controllers  ability  to  maintain  situation  awareness? 

Some  other  questions  include  how  to  resectorize?  Should  airspace  boundaries  be  set  up  in  such  a 
way  as  to  offer  an  infinite  number  of  configuration  capabilities  (rubber  band  or  jello  sectors),  or 
should  resectorization  occur  within  predefined  regions  of  airspace?  Incremental  airspace 
resectorization  within  a  specified  period  of  time  may  not  be  as  easy  for  the  controller  to  maintain 
his/her  comfort  level,  especially  in  an  airspace  with  a  high  degree  of  complexity.  Predefined 
regions  of  airspace  offer  advantages  in  terms  of  training  and  simplicity.  However,  these  defined 
regions  may  work  well  with  aircraft  flying  fixed  routes,  but  what  about  when  they  are  on  wind 
optimized  or  user-preferred  routes?  Other  questions  consider  the  changes  in  procedures, 
automation,  and  communications  that  need  to  be  addressed  and  may  only  be  able  to  be  examined 
in  simulation.  For  example,  what  role  does  commonly-used  types  of  procedures  (i.e.,  crossing 
restrictions)  play  in  structuring  traffic  and  reducing  complexity?  How  does  using  such 
procedures  constrain  dynamic  resectorization?  Under  what  conditions  might  dynamic 
resectorization  result  in  the  need  to  revise  or  create  a  procedure?  If/when  revised  or  new 
procedures  are  needed  to  complement  a  dynamic  resectorization,  how  are  they  redefined  and 
distributed  dynamically?  Under  what  conditions  can  dynamic  resectorization  remove  the  need 
for  an  active  flow  restriction,  or  what  conditions  might  dynamic  resectorization  result  in  the  need 
to  revise  or  create  a  traffic  flow  restriction? 

Furthermore,  what  are  the  automation  requirements  to  support  dynamic  resectorization?  All 
messages,  including  text  messages  and  display  commands,  must  be  rerouted  at  the  moment  of 
airspace  change  activation.  If  flight  strips  are  still  in  use,  any  reposting  and  reprinting  of  strips 
must  be  controlled  to  ensure  appropriately  posted  strips  and  minimum  disruption  on  the  strip 
board.  Automated  hand-off  capabilities  must  be  equal,  and  support  tools,  such  as  conflict  probe. 
Traffic  Management  Advisor,  and  datalink  need  to  be  supported.  The  number  of  sector 
configurations  must  be  supported  by  computer  capacity,  number  of  sector  displays,  and  number 
of  operational  positions. 

The  present  study  identified  specific  high  density  traffic  and  weather  situations  that  can  benefit 
from  dynamic  resectorization.  Yet,  dynamic  resectorization  may  not  be  effective  for  all  traffic 
situations.  It  is  important  to  identify  situations  where  dynamic  resectorization  may  be  beneficial 
and  situations  where  resectorization  should  not  be  made. 
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Appendix  A 


INFORMED  CONSENT 

/, _ _ _ ,  understand  that  this  study,  entitled  "Inter- Facility 

Boundary  Adjustment”  is  sponsored  by  the  Federal  Aviation  Administration  and  is  being 
directed  by  Dr.  Earl  Stein. 

Nature  and  Purpose: 

I  have  been  recruited  to  volunteer  as  a  participant  in  the  project  named  above.  The  purpose  of 
this  study  is  to  conduct  a  human  factors  evaluation  of  the  potential  impact  on  adjacent  center 
controllers  operating  in  adaptive  sectors  using  real-time  air  traffic  control  (ATC)  simulation. 
This  study  will  compare  operations  between  a  dynamic  en  route  airspace  to  using  a  standard  en 
route  airspace  with  fixed  boundaries.  The  research  team  will  use  the  data  gathered  from  this 
simulation  to  provide  input  on  the  viability  of  developing  and  implementing  inter-facility 
adaptive  en  route  sector  boundaries. 

Experimental  Procedures: 

Participants  will  conduct  traffic  in  two  different  experimental  conditions  for  this  simulation.  In 
the  first  condition,  controllers  will  employ  a  boundary  adjustment  between  the  two  Centers.  The 
second  condition  will  involve  current  operating  procedures  for  controlling  and  directing  traffic 
between  Center  facilities,  and  will  serve  as  a  baseline  for  comparison.  Each  day  of  the 
simulation,  controllers  will  work  four  different  traffic  scenarios  with  realistic  traffic  levels  for  an 
en  route  sector.  Two  of  the  scenarios  will  consist  of  rather  heavy  traffic  and  two  will  be  a 
combination  of  medium  traffic  accompanied  by  a  severe  weather  system. 

An  automated  data  collection  system  will  record  important  simulation  events  and  produce  a  set 
of  system  effectiveness  measures,  which  include  safety,  capacity,  efficiency,  and  controller 
workload.  In  addition,  SATCSs  will  make  over-the-shoulder  observations  to  evaluate  controller 
effectiveness  while  operating  in  an  adaptive  airspace.  After  each  scenario,  controllers  will 
complete  questionnaires  to  evaluate  the  benefits  of  boundary  adjustments  between  facilities.  The 
simulation  will  be  audio-video  recorded  for  the  purposes  of  post  experiment  content  analysis  of 
controller  communications. 

Discomfort  and  Risks: 

I  understand  that  I  will  not  be  exposed  to  any  foreseeable  risks  or  intrusive  measurement 
techniques. 

Benefits: 

I  understand  that  the  only  benefit  to  me  is  that  I  will  have  the  opportunity  to  provide  feedback 
and  valuable  insight  on  the  feasibility  of  Inter-Facility  Boundary  Adjustment  to  the  research 
team  conducting  the  simulation. 
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Subject  Responsibilities: 

I  am  aware  that  to  participate  in  this  study  that  I  am  required  to  have  20/30  normal  or  corrected- 
to-normal  vision,  and  not  to  be  on  any  medical  waiver. 

Compensation  and  Injury: 

I  agree  to  immediately  report  any  injury  or  suspected  adverse  effect  to  Dr.  Randy  Sollenberger  at 
(609)-485-7169.  Local  clinics  and  hospitals  would  provide  any  necessary  treatment.  I  agree  to 
provide,  if  requested,  copies  of  all  insurance  and  medical  records  arising  from  any  such  care  for 
injuries/medical  problems. 

Subject's  Assurances: 

I  understand  that  my  participation  in  this  study  is  completely  voluntary.  I  am  participating 
because  I  want  to.  Dr.  Sollenberger  has  adequately  answered  any  and  all  questions  I  have  about 
this  study,  my  participation,  and  the  procedures  involved.  I  understand  that  Dr.  Sollenberger  will 
be  available  to  answer  any  questions  concerning  procedures  throughout  this  study. 

I  understand  that  if  new  findings  develop  during  the  course  of  this  research  that  may  relate  to  my 
decision  to  continue  participation,  I  will  be  informed. 

I  have  not  given  up  any  of  my  legal  rights  or  released  any  individual  or  institution  from  liability 
for  negligence. 

Participation  in  this  experiment  is  strictly  voluntary  and  I  have  the  freedom  to  withdraw  at  any 
time  without  penalty.  I  also  understand  that  the  researcher  of  this  study  may  terminate  my 
participation  if  he  feels  this  to  be  in  my  best  interest.  My  participation  is  strictly  confidential, 
and  no  individual  names  or  identities  will  be  recorded  or  released  in  any  reports. 

If  I  have  questions  about  this  study  or  need  to  report  any  adverse  effects  from  the  research 
procedures,  I  will  contact  Dr.  Sollenberger  at  609-485-7 169(w)  609-476-2745(h). 

Signature  Lines: 

I  have  read  this  consent  document.  /  understand  its  contents,  and  I  freely  consent  to  participate 
in  this  study  under  the  conditions  described.  I  have  received  a  copy  of  this  consent  form. 

Research  Participant: _  Date: _ 

Investigator: _ _  Date: _ 


Witness: 


Date: 


Appendix  B 


Scenario  Presentation  Order 


Controller 

1st 

2nd 

3rd 

4th 

5th 

6th 

7th 

8th 

SOI 

RZHD-A ( 1 ) 

STWx- A { 1 ) 

STHD-A (1) 

RZWx-A(l) 

RZHD-B (1) 

STWx-B (1) 

STHD-B (1) 

RZWx-B (1) 

S02 

RZHD-B (2 ) 

STWx-B (2) 

STHD-B (2) 

RZWx-B  (2) 

RZHD-A (2) 

STWx-A (2) 

STHD-A (2) 

RZWx-A (2) 

S03 

STHD-A(l) 

STWx-A ( 1 ) 

RZHD-A (1) 

RZWx-A ( 1 ) 

STHD-B (1) 

STWx-B (1) 

RZHD-B (1) 

RZWx-B (1) 

S04 

STHD-B (2 ) 

STWx-B (2) 

RZHD-B (2) 

RZWx-B (2) 

STHD-A(2) 

STWx-A (2) 

RZHD-A (2) 

RZWx-A (2) 

S05 

STHD-A(l) 

RZWx-A(l) 

RZHD-A (1) 

STWx-A (1) 

STHD-B (1) 

RZWx-B ( 1 ) 

RZHD-B (1) 

STWx-B (1) 

S06 

STHD-B (2) 

RZWx-B (2) 

RZHD-B (2) 

STWx-B (2) 

STHD-A ( 2 ) 

RZWx-A (2) 

RZHD-A (2) 

STWx-A ( 2 ) 

S07 

STWx -A ( 1 ) 

STHD-A { 1 ) 

RZWx-A ( 1 ) 

RZHD-A (1) 

STWx-B (1) 

STHD-B (1) 

RZWx-B (1) 

RZHD-B (1) 

S08 

STWx-B (2 ) 

STHD-B (2) 

RZWx-B (2) 

RZHD-B (2) 

STWx-A ( 2 ) 

STHD-A (2) 

RZWx-A (2) 

RZHD-A ( 2 ) 

S09 

RZHD-A (1) 

RZWx-A(l) 

STHD-A (!) 

STWx-A (1) 

RZHD-B (1) 

RZWx-B (1) 

STHD-B (1) 

STWx-B (1) 

S10 

RZHD-B (2) 

RZWx-B (2) 

STHD-B (2) 

STWx-B ( 2 ) 

RZHD-A (2 ) 

RZWx-A (2) 

STHD-A (2) 

STWx-A ( 2 ) 

Sll 

RZWx-A(l) 

RZHD-A (1) 

STWx-A ( 1 ) 

STHD-A (1) 

RZWx-B { 1 ) 

RZHD-B (1) 

STWx-B ( 1 ) 

STHD-B (1) 

S12 

RZWx-B (2 ) 

RZHD-B (2) 

STWx-B (2) 

STHD-B (2) 

RZWx-A (2) 

RZHD-A (2) 

STWx-A (2) 

STHD-A (2) 

Note:  The  designators  STHD  and  STWx  indicate  baseline  airspace  boundaries  for  the 
high-density  scenario  and  weather  scenario,  respectively.  The  designators  RZHD  and 
RZWx  indicate  dynamic  resectorization  for  the  high-density  scenario  and  weather 
scenario,  respectively.  The  letters  A  and  B  indicate  ZNO  airspace  and  ZSO  airspace, 
respectively.  The  numbers  (1)  and  (2)  identify  the  SMEs  observing  the  controllers. 
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Appendix  C 


Subject  Matter  Expert  Observer  Rating  Form 


Observer  Code _  Date _ 

INSTRUCTIONS 

This  form  is  designed  to  be  used  by  supervisory  air  traffic  control  specialists  to  evaluate  the 
effectiveness  of  controllers  working  in  simulation  environments.  SATCSs  will  observe  and  rate 
the  performance  of  controllers  in  several  different  performance  dimensions  using  the  scale  below 
as  a  general  purpose  guide.  Use  the  entire  scale  range  as  much  as  possible.  You  will  see  a  wide 
range  of  controller  performance.  Take  extensive  notes  on  what  you  see.  Do  not  depend  on  your 
memory.  Write  down  your  observations.  Space  is  provided  after  each  scale  for  comments.  You 
may  make  preliminary  ratings  during  the  course  of  the  scenario.  However,  wait  until  the 
scenario  is  finished  before  making  your  final  ratings  and  remain  flexible  until  the  end  when  you 
have  had  an  opportunity  to  see  all  the  available  behavior.  At  all  times,  please  focus  on  what  you 
actually  see  and  hear.  This  includes  what  the  controller  does  and  what  you  might  reasonably 
infer  from  the  actions  of  the  pilots.  Try  to  avoid  inferring  what  you  think  may  be  happening.  If 
you  do  not  observe  relevant  behavior  or  the  results  of  that  behavior,  then  you  may  leave  a 
specific  rating  blank.  Also,  please  write  down  any  comments  that  may  help  improve  this 
evaluation  form.  Do  not  write  your  name  on  the  form  itself.  Your  identity  will  remain 
anonymous,  as  your  data  will  be  identified  by  an  code  known  only  to  yourself  and  the 
researchers  conducting  this  study.  The  observations  you  make  do  not  need  to  be  restricted  to  the 
performance  areas  covered  in  this  form  and  may  include  other  areas  that  you  think  are  important. 

ASSUMPTIONS 

ATC  is  a  complex  activity  that  contains  both  observable  and  unobservable  behavior.  There  are 
so  many  complex  behaviors  involved  that  no  observational  rating  form  can  cover  everything.  A 
sample  of  the  behaviors  is  the  best  that  can  be  achieved,  and  a  good  form  focuses  on  those 
behaviors  that  controllers  themselves  have  identified  as  the  most  relevant  in  terms  of  their  overall 
performance.  Most  controller  performance  is  at  or  above  the  minimum  standards  regarding 
safety  and  efficiency.  The  goal  of  the  rating  system  is  to  differentiate  performance  above  this 
minimum.  The  lowest  rating  should  be  assigned  for  meeting  minimum  standards  and  also  for 
anything  below  the  minimum  since  this  should  be  a  rare  event.  It  is  important  for  the 
observer/rater  to  feel  comfortable  using  the  entire  scale  and  to  understand  that  all  ratings  should 
be  based  on  behavior  that  is  actually  observed. 
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Rating  Scale  Descriptors 


Remove  this  Page  and  keep  it  available  while  doing  ratings 


-  jM  i;-  •  ] 

!  |  |  ’  ■  «  f'oKSvi'-  i 

1 

Least  Effective 

Unconfident,  Indecisive,  Inefficient, 
Disorganized,  Behind  the  power  curve,  Rough, 
Leaves  some  tasks  incomplete,  Makes 
mistakes 

2 

Poor 

May  issue  conflicting  instructions,  Doesn’t 
plan  completely 

3 

Fair 

Distracted  between  tasks 

4 

Low  Satisfactory 

Postpones  routine  actions 

5 

High  Satisfactory 

Knows  the  job  fairly  well 

6 

Good 

Works  steadily,  Solves  most  problems 

7 

Very  Good 

Knows  the  job  thoroughly,  Plans  well 

8 

Most  Effective 

Confident,  Decisive,  Efficient,  Organized, 

Ahead  of  the  power  curve.  Smooth,  Completes 
all  necessary  tasks,  Makes  no  mistakes 
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I  -  Maintaining  Safe  and  Efficient  Traffic  Flow 


1.  Maintaining  Separation  and  Resolving  Potential  Conflicts .  1  2  3  4  5  6  7  8 

•  using  control  instructions  that  maintain  appropriate  aircraft 

and  airspace  separation 

•  detecting  and  resolving  impending  conflicts  early 

•  recognizing  the  need  for  speed  restrictions  and  wake 

turbulence  separation 

Comments: 


2.  Sequencing  Aircraft  Efficiently . 12345678 

•  using  efficient  and  orderly  spacing  techniques  for  arrival, 

departure,  and  en  route  aircraft 

•  maintaining  safe  arrival  and  departure  intervals  that  minimize 

delays 

Comments: 


3.  Using  Control  Instructions  Effectively/Efficiently .  1  2  3  4  5  6  7  8 

•  providing  accurate  navigational  assistance  to  pilots 

•  issuing  economical  clearances  that  result  in  need  for  few 

additional  instructions  to  handle  aircraft  completely 

•  ensuring  clearances  use  minimum  necessary  flight  path 

changes 

Comments: 


4.  Overall  Safe  and  Efficient  Traffic  Flow  Scale  Rating .  1  2  3  4  5  6  7  8 

Comments: 
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II  -  Maintaining  Attention  and  Situation  Awareness 


5.  Maintaining  Awareness  of  Aircraft  Positions .  12345678 

•  avoiding  fixation  on  one  area  of  the  radar  scope  when  other 

areas  need  attention 

•  using  scanning  patterns  that  monitor  all  aircraft  on  the  radar 

scope 

Comments: 


6.  Ensuring  Positive  Control .  12345678 

•  tailoring  control  actions  to  situation 

•  using  effective  procedures  for  handling  heavy,  emergency,  and 

unusual  traffic  situations 

Comments: 


7.  Detecting  Pilot  Deviations  from  Control  Instructions .  12345678 

•  ensuring  that  pilots  follow  assigned  clearances  correctly 

•  correcting  pilot  deviations  in  a  timely  manner 

Comments: 


8.  Correcting  Own  Errors  in  a  Timely  Manner .  12345678 

•  acting  quickly  to  correct  errors 

•  changing  an  issued  clearance  when  necessary  to  expedite 

traffic  flow 

Comments: 


9.  Overall  Attention  and  Situation  Awareness  Scale  Rating .  1  2  3  4  5  6  7  8 

Comments: 
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in  -  Prioritizing 


10.  Taking  Actions  in  an  Appropriate  Order  of  Importance . 1  2  3  4  5  6  7  8 

•  resolving  situations  that  need  immediate  attention  before 

handling  low  priority  tasks 

•  issuing  control  instructions  in  a  prioritized,  structured,  and 

timely  manner 

Comments: 


1 1 .  Preplanning  Control  Actions . 12345678 

•  scanning  adjacent  sectors  to  plan  for  future  and  conflicting 

traffic 

•  studying  pending  flight  strips  in  bay 
Comments 


12.  Handling  Control  Tasks  for  Several  Aircraft .  1  2  3  4  5  6  7  8 

•  shifting  control  tasks  between  several  aircraft  when  necessary 

•  communicating  in  timely  fashion  while  sharing  time  with 

other  actions 

Comments: 


13.  Marking  Flight  Strips  while  Performing  Other  Tasks .  1  2  3  4  5  6  7  8 

•  marking  flight  strips  accurately  while  talking  or  performing 

other  tasks 

•  keeping  flight  strips  current 
Comments: 


14.  Overall  Prioritizing  Scale  Rating .  1  2  3  4  5  6  7  8 

Comments: 
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IV  -  Providing  Control  Information 


15.  Providing  Essential  Air  Traffic  Control  Information .  1  2  3  4  5  6  7  8 

•  providing  mandatory  services  and  advisories  to  pilots  in  a 

timely  manner 

•  exchanging  essential  information 
Comments: 

16.  Providing  Additional  Air  Traffic  Control  Information .  1  2  3  4  5  6  7  8 

•  providing  additional  services  when  workload  is  not  a  factor 

•  exchanging  additional  information 

Comments: 

17.  Providing  Coordination .  12345678 

•  providing  effective  and  timely  coordination 

•  using  proper  point-out  procedures 

Comments: 

18.  Overall  Providing  Control  Information  Scale  Rating . 1  2  3  4  5  6  7  8 

Comments: 
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V  -  Technical  Knowledge 


19.  Showing  Knowledge  of  LO As  and  SOPs . 1  2  3  4  5  6  7  8 

•  controlling  traffic  as  depicted  in  current  LOAs  and  SOPs 

•  performing  handoff  procedures  correctly 

Comments: 


20a.  Showing  Knowledge  of  Aircraft  Capabilities  and  Limitations .  12345678 

•  using  appropriate  speed,  vectoring,  and/or  altitude 

assignments  to  separate  aircraft  with  varied  flight  capabilities 

•  issuing  clearances  that  are  within  aircraft  performance 

parameters 

Comments: 


20b.  Showing  Effective  Use  of  Equipment . 1  2  3  4  5  6  7  8 

•  updating  data  blocks 

•  using  equipment  capabilities 

Comments: 


21.  Overall  Technical  Knowledge  Scale  Rating .  1  2  3  4  5  6  7  8 

Comments: 
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VI  -  Communicating 


22.  Using  Proper  Phraseology .  12345678 

•  using  words  and  phrases  specified  in  the  71 10.65 

•  using  phraseology  that  is  appropriate  for  the  situation 

•  using  minimum  necessary  verbiage 

Comments: 


23.  Communicating  Clearly  and  Efficiently .  12345678 

•  speaking  at  the  proper  volume  and  rate  for  pilots  to  understand 

•  speaking  fluently  while  scanning  or  performing  other  tasks 

•  ensuring  clearance  delivery  is  complete,  correct  and  timely 

•  speaking  with  confident,  authoritative  tone  of  voice 

Comments: 


24.  Listening  to  Pilot  Readbacks  and  Requests .  12345678 

•  correcting  pilot  readback  errors 

•  acknowledging  pilot  or  other  controller  requests  promptly 

•  processing  requests  correctly  in  a  timely  manner 

Comments: 


25.  Overall  Communicating  Scale  Rating .  12345678 

Comments: 
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Questionnaires 

BACKGROUND  QUESTIONNAIRE 

Participant  Code _  Date 

INSTRUCTIONS 


This  questionnaire  is  designed  to  obtain  information  about  your  background  as  an  air  traffic 
control  specialist.  The  information  will  be  used  to  describe  the  participants  in  this  study  as  a 
group  in  written  and/or  oral  reports.  Your  identity  will  remain  anonymous,  so  do  not  write  your 
name  on  the  form.  Instead,  your  data  will  be  identified  by  a  participant  code  known  only  to 
yourself  and  the  researchers  conducting  this  study. 


Sex:  □  male  □  female 

1 .  What  is  your  job  position  or  title? 


2.  What  is  your  age? 

_ years 

3.  How  many  years  have  you  worked  as  an  air  traffic  control  specialist? 
_ years 

4.  How  many  years  have  you  been  a  Full  Performance  Level  controller? 
_ years 

5.  How  many  of  the  past  12  months  have  you  actively  controlled  traffic? 
_ months 

6.  Please  briefly  describe  your  air  traffic  control  training  and  experience. 
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POST-SCENARIO  QUESTIONNAIRE 


Participant  Code _  Date _ 

Scenario  Code _ 

1 .  Please  circle  the  number  that  best  describes  how  well  you  controlled  traffic  during  this 


scenario. 

extremely  1 

2 

3 

4 

5 

6 

7 

8 

9 

10  extremely 

poor 

Comments 

well 

2.  Please  circle  the  number  that  best  describes  your  overall  physical  and  mental  workload 
during  this  scenario. 

extremely  1  2345678910  extremely 

low  high 

Comments _ 


3.  Please  circle  the  number  that  best  describes  your  overall  situational  awareness  during  this 
scenario. 

extremely  1  2345678910  extremely 

low  high 

Comments _ 


4.  Please  circle  the  number  that  best  describes  how  your  overall  coordination  was  affected 
during  this  scenario. 

very  1  23456789  10  A  great 

little  deal 

Comments _ 
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p 


5.  Please  circle  the  number  that  best  describes  how  realistic  the  traffic  flow  in  this  scenario 
was. 

extremely  1  2  3  4  5  6  7  8  910  extremely 

realistic  unrealistic 

Comments _ 


6.  Please  circle  the  number  that  best  describes  how  difficult  this  scenario  was. 

extremely  1  2  3  45  67  8  910  extremely 

easy  difficult 

Comments _ 


7.  Please  circle  the  number  that  best  describes  how  well  the  simulation-pilots  performed 
during  this  scenario. 

extremely  1  2  3  4  5  6  7  8  910  extremely 

poor  well 

Comments  _ 


8.  Do  you  have  any  other  comments  about  your  experiences  during  this  scenario? 
Comments _ 
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COMPLETE  THIS  LAST  SECTION  ONLY  IF  THE  SCENARIO  YOU  JUST  FINISHED  HAD 
A  BOUNDARY  ADJUSTMENT 

THESE  QUESTIONS  FOCUS  ON  THE  TIME  IN  THE  SCENARIO  WHEN  THE 
BOUNDARY  ADJUSTMENT  WAS  IMPLEMENTED 

9.  During  the  time  of  the  boundary  adjustment,  what  was  the  effect  on  your  performance? 

extremely  1  2345678910  extremely 

negative  positive 

Comments _ 


10.  During  the  time  of  the  boundary  adjustment,  what  was  the  effect  on  your  physical/mental 
workload? 

extremely  1  2  3  4  5  6  7  8  910  extremely 

negative  positive 

Comments _ 


1 1 .  During  the  time  of  the  boundary  adjustment,  how  difficult  was  it  to  maintain  your 
situational  awareness? 

Not  very  1  23456789  10  extremely 

difficult  difficult 

Comments _ 


12.  During  the  time  of  the  boundary  adjustment,  to  what  extent  was  your  coordination 
affected? 

very  1  23456789  10  A  great 

little  deal 

Comments _ 
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NASA  Task  Load  Index 


Participant  Code _  Date _ 

Scenario  Code _ 

Mental  Demand 

How  much  mental  and  perceptual  activity  is  required  (e.g.,  thinking,  deciding,  calculating, 
remembering,  looking,  searching)?  Is  the  task  easy  or  demanding,  simple  or  complex? 

Circle  the  number  that  best  describes  the  mental  demand  during  this  scenario. 

extremely  1234567  8  9  10  extremely 

low  high 

Physical  Demand 

How  much  physical  activity  is  required  (e.g.,  pushing,  turning,  controlling,  activating)?  Is 
the  task  easy  or  demanding,  slow  or  brisk,  slack  or  strenuous? 

Circle  the  number  that  best  describes  the  physical  demand  during  this  scenario, 
extremely  12  3  45  67  8  9  10  extremely 

low  high 

Temporal  Demand 

How  much  time  pressure  do  you  feel  due  to  the  rate  or  pace  at  which  the  task 
occurred?  Is  the  pace  slow  and  leisurely  or  rapid  and  frantic. 

Circle  the  number  that  best  describes  the  temporal  demand  during  this  scenario, 
extremely  12  3  45  67  8  9  10  extremely 

low  high 
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Performance 


How  successful  do  you  think  you  are  in  accomplishing  the  goals  of  the  task?  How  satisfied  are 
you  with  your  performance  in  accomplishing  these  goals? 

Circle  the  number  that  best  describes  your  performance  during  this  scenario. 


extremely  12345678 

9 

10 

extremely 

low 

high 

Effort 

How  hard  did  you  have  to  work  (mentally  and  physically)  to  accomplish  this  level  of 
performance? 

Circle  the  number  that  best  describes  your  effort  during  this  scenario, 
extremely  12345678  9  10  extremely 

low  high 

Frustration 

How  insecure,  discouraged,  irritated,  stressed,  and  annoyed  versus  secure,  gratified,  content, 
relaxed,  and  complacent  do  you  feel  in  performing  the  task? 

Circle  the  number  that  best  describes  your  level  of  frustration  during  this  scenario. 

extremely  12345  67  8  9  10  extremely 

low  high 
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EXIT  QUESTIONNAIRE 


Participant  Code. 


Date 


INSTRUCTIONS 

The  purpose  of  this  questionnaire  is  to  obtain  feedback  from  you  concerning  different  aspects  of 
the  experiment.  This  information  will  be  used  to  improve  our  simulation  in  the  future.  In 
addition  to  your  ratings,  you  will  be  asked  to  make  comments  on  some  of  the  questions.  Even  if 
your  ratings  are  other  than  favorable,  you  may  wish  to  make  further  comments.  If  you  feel  you 
have  any  helpful  ideas  regarding  this  experiment,  we  would  like  to  hear  from  you.  So  that  your 
identity  can  remain  anonymous,  your  actual  name  should  not  be  written  on  this  form.  Instead, 
your  data  will  be  identified  by  a  participant  code  known  only  to  yourself  and  the  experiments. 


1.  In  general,  how  realistic  was  the  simulation? 

Not  very  123456789  10  Extremely 

realistic  realistic 

Comments _ _ _ _ _ _ _ — 


2.  To  what  extent  did  the  ATWIT  probe  technique  interfere  with  your  performance? 

Not  very  1  2  3  4  5  6  7  8  9  10  A  great 

much  deal 

Comments _ _ _ _ _ 


3.  While  you  were  controlling  traffic  in  ZNO,  what  type  of  impact  did  the  boundary  adjustment 
have  on  your  performance? 

very  1  2  3  4  5  6  7  8  9  10  very 

negative  positive 

Comments _ _ _ _ _ 
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4.  While  you  were  controlling  traffic  in  ZSO,  what  type  of  impact  did  the  boundary  adjustment 
have  on  your  performance? 

very  1  23456789  10  very 

negative  positive 

Comments _ 


5.  Circle  the  number  that  best  describes  the  feasibility  of  Inter-Facility  boundary  adjustment  in 
the  National  Airspace  System  (NAS). 

Not  very  1  23456789  10  extremely 

feasible  feasible 

Comments _ 


6.  Describe  any  situation  or  conditions  where  boundary  adjustment  would  be  useful  in  your 
facility. 

Comments _ 


7.  What  do  you  see  as  the  benefits  of  this  concept? 
Comments _ 
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Inter-Facility  Boundary  Adjustment  Policies  and  Procedures 

Adjusting  airspace  between  the  North  ARTCC  and  South  ARTCC  (ZNO  and  ZSO)  will  involve 
two  pre-defmed  regions  within  each  facility  that  are  stored  in  the  host  system.  Flight  data 
processing  will  be  automated  to  coincide  with  airspace  boundary  adjustments.  The  operational 
advantages  stemming  from  this  concept  may  include 

1.  reduced  controller  workload, 

a.  reduced  coordination  activities 

b.  more  balanced  traffic 

2.  greater  user  flexibility, 

a.  increased  preferential  routing 

b.  decreased  fuel  bum 

3.  reduction  in  delays, 

4.  reduced  flow  restrictions,  miles  in  trail,  and 

5.  increased  safety. 

Ideally,  the  Traffic  Management  Unit  (TMU)  is  the  first  to  recognize  the  potential  need  for  a 
boundary  adjustment.  For  the  purposes  of  this  simulation,  the  resectorization  process  will  be 
driven  by  the  SMEs  who  are  acting  as  the  local  TMU.  The  TMU,  using  current  and  projected 
ETMS  and/or  NWS/WARP  data,  will  determine  the  need  for  a  boundary  adjustment  at  a  specific 
time.  Timely  decisions  for  resectorization  are  important.  It  is  critical  that  the  decision  to 
implement  a  resectorization  be  made  in  a  timely  manner.  The  actual  events  that  trigger  the 
decision  are  based  on  known  traffic/weather  trends  and  projected  data  (time  of  year,  special 
events,  SUA  activation,  etc.).  Using  the  equipment  available,  the  specialist  calculates  fix  loads 
and  projected  track  data  to  determine  the  most  operationally  viable  time  for  boundary 
adjustment.  A  late  call  can  result  in  sector  complexity  that  ultimately  leads  to  a  loss  of  situation 
awareness  for  the  controller(s)  working  the  reconfigured  airspace.  An  early  call,  however,  is  an 
inefficient  use  of  resources. 

The  two  scenarios  used  in  this  study  are  airspace  boundary  adjustment  for  weather  deviations 
and  an  airspace  boundary  adjustment  for  traffic  density/balancing.  In  the  traffic  density  scenario, 
the  TMU  recognizes  the  need  to  resectorize  due  to  a  large  amount  of  aircraft  projected  in  ZNO 
along  J 1 .  The  facility  TMU  notifies  the  area  supervisors  for  ZNO  and  ZSO.  ATCSCC,  with 
input  from  the  local  TMU,  makes  the  call  on  the  exact  time  to  resectorize.  All  parties  must 
concur  prior  to  boundary  reconfiguration.  When  a  boundary  adjustment  is  initiated  (ZNO 
allocating  the  Northeast  Corridor  to  ZSO),  the  sector  boundaries  will  reconfigure  on  all  displays. 
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Controllers  on  the  sectors  receive  2  min  prior  notification  in  this  simulation.  All  hand-offs  and 
communication  transfers  shall  be  accomplished  during  this  transition  period. 

The  weather  deviation  scenario  is  implemented  in  the  same  manner.  For  this  simulation,  local 
weather  patterns  dictated  a  region  of  airspace  in  ZSO  (Thunder  Alley)  to  be  allocated  to  ZNO  in 
order  to  accommodate  deviating  traffic  along  J2,  J10,  and  J20.  The  local  TMU  becomes  aware 
of  severe  weather  buildup  and  makes  the  call  before  traffic  deviations  increase  workload  due  to 
the  large  amount  of  coordination  required.  The  allocation  of  Thunder  Alley  to  ZNO  allows  the 
aircraft  that  are  in  close  proximity  to  the  weather  cells  to  remain  in  ZNO  (the  originating 
facility).  This  reduces  the  multiple  handoffs  and  point  outs  required  as  aircraft  will  no  longer  be 
deviating  into  ZSO  airspace.  As  with  the  high-density  scenario,  the  controllers  on  the  sectors 
receive  2  min  prior  notification  to  complete  hand-offs  and  communication  transfers. 
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